• 7 min read

Four missing env vars and a 500 error

A client's Redmine instance was throwing 500 errors every time someone tried to upload a file. The UI gave no details — just "Internal Server Error." Wiki pages, issues, everything else worked fine. But attach a photo? Boom.

Here's how the diagnosis went, step by step, and what it took to fix it.

The symptoms

Tierra de Fe Viva 2026 — a Redmine instance running on Coolify (self-hosted PaaS). Everything worked except file uploads. Every attachment request returned HTTP 500.

The client had been living with it. "Just can't upload images" — until someone needed it to work now.

Step 1: find the logs

First rule of debugging: read the actual error, don't guess.

docker logs redmine-r0swock88sogkg44k0040g4s --tail 200 2>&1 \
  | grep -i -E 'error|upload|attach|file|500'

The output was immediate:

[24177ce8] Started POST "/uploads.js?attachment_id=1&filename=MAR%202.jpg&content_type=image%2Fjpeg"
[24177ce8] Completed 500 Internal Server Error in 1008ms
[24177ce8] Aws::Errors::MissingRegionError (No region was provided.
  Configure the `:region` option or export the region name to ENV['AWS_REGION']):
[24177ce8] plugins/redmica_s3/lib/redmica_s3/attachment_patch.rb:42:in `block in disk_filename'

There it was. The redmica_s3 plugin was intercepting every file upload and trying to send it to S3 — but it couldn't because region was nil.

The error chain:

  1. User uploads file → AttachmentsController#upload
  2. redmica_s3 patches Attachment to redirect storage to S3
  3. AWS SDK tries to establish connection
  4. It attempts instance metadata at 169.254.169.254:80 (AWS IAM) — timeout
  5. Falls back to environment variables — all empty
  6. Aws::Errors::MissingRegionError — 500

Step 2: trace the configuration

The plugin reads from config/s3.yml:

production:
    access_key_id: <%= ENV['S3_ACCESS_KEY_ID'] %>
    secret_access_key: <%= ENV['S3_SECRET_ACCESS_KEY'] %>
    bucket: <%= ENV['S3_BUCKET'] %>
    region: <%= ENV['S3_REGION'] %>
    folder: public/galeria

Clean ERB templating — reads from environment variables. So I checked the container:

docker exec redmine-... env | grep S3_
# (nothing)

Zero S3 environment variables. The config file existed, the plugin was loaded, but the credentials were never injected into the container.

Step 3: where were the credentials supposed to come from?

The project uses GitLab CI/CD to build and deploy. Checking the CI variables:

glab api projects/80576272/variables | jq '.[] | .key + "=" + .value[:20]'

Result:

S3_BUCKET=tierradefeviva2026
S3_REGION=eu-central-1
S3_SECRET_ACCESS_KEY=*** (masked)
S3_ACCESS_KEY_ID=(hidden, masked)

All four variables were defined in GitLab CI. But the deploy stage only called Coolify's /deploy webhook — a simple POST that triggers a pull-and-restart. It didn't inject environment variables into the compose definition.

The gap: GitLab CI variables exist during pipeline execution. Coolify compose definitions are persistent. These are two different configuration planes, and nothing bridged them.

Step 4: fix the right layer

Two options:

  1. Modify the CI/CD pipeline to update Coolify's compose with the S3 vars on each deploy — complex, fragile, mixes concerns
  2. Add the vars directly to Coolify's compose — they're secrets that rarely change, they belong in the infrastructure config

Option 2 was the right call. These are AWS credentials for a specific bucket — infrastructure configuration, not application code.

I updated the Coolify service's docker_compose_raw via API to add four environment variables to the redmine service, then restarted it.

Step 5: verify (for real this time)

"Works on my machine" isn't good enough. I verified three ways:

1. Container has the vars:

docker exec redmine-... env | grep S3_
# S3_BUCKET=tierradefeviva2026
# S3_ACCESS_KEY_ID=AKIAR...YE5M
# S3_SECRET_ACCESS_KEY=***
# S3_REGION=eu-central-1

2. S3 connectivity from inside the container:

require 'aws-sdk-s3'
s3 = Aws::S3::Resource.new(region: ENV['S3_REGION'], ...)
bucket = s3.bucket(ENV['S3_BUCKET'])
bucket.exists?  # => true

3. Actual upload test:

obj = bucket.object('public/galeria/test_connection.txt')
obj.put(body: 'S3 upload test from Redmine', content_type: 'text/plain')
# => Upload successful!

Then I cleaned up the test file. Uploads work.

The lesson

This was a configuration drift problem. The code (Dockerfile, s3.yml, plugin) was ready. The CI variables existed. But nobody connected the two at the infrastructure level.

Three things to remember:

  1. Read the logs first. The error message told us exactly what was wrong — MissingRegionError. No need to speculate about permissions, file sizes, or Rails versions.
  2. Know your configuration planes. GitLab CI variables ≠ container environment variables. If your app reads from ENV, the vars must be in the container runtime, not just in the pipeline that builds it.
  3. Verify end-to-end, not just "it restarted." A 200 from your deploy API doesn't mean the app works. Upload a file. Read from S3. Confirm the actual user-facing feature.

A 500 that looked like a mystery was just four missing environment variables.

← Back to Blog