← Back to all stories

GitHub Actions Secrets: The Time I Almost Leaked Our API Keys

A cautionary tale about debugging in production, log outputs, and why you should always use secret masking.

It was a simple debugging line. Just a quick echo to see what was happening. What could go wrong?

🤦 The Mistake

Our deployment was failing with a cryptic error. In my frustration, I added what I thought was a harmless debug line:

workflow.yml (DON'T DO THIS)
- name: Debug environment
  run: |
    echo "Debugging deployment..."
    echo "AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}"
    env

I committed it, pushed, and went to grab coffee while the workflow ran.

😰 What Actually Happened

GitHub Actions is smart - it masks secrets in logs. When I looked at the output, I saw:

Log output
Debugging deployment...
AWS_ACCESS_KEY_ID: ***

But wait. The env command at the end? It printed ALL environment variables. And while the secret itself was masked, I had also exported a variable earlier that contained a partial key that wasn't registered as a secret:

The dangerous output
AWS_ACCESS_KEY_PREFIX=AKIA...
ENCODED_CREDENTIALS=base64_encoded_stuff_here
⚠️
The lesson: GitHub only masks values it knows are secrets. Derived values, encoded values, or partial values won't be automatically masked.

🔧 The Fix

First, I rotated ALL potentially exposed credentials immediately. Then I implemented proper secret handling:

workflow.yml (the safe way)
- name: Deploy
  run: ./deploy.sh
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  # Never echo secrets, never run 'env' in debug

✅ Best Practices

🚫

Never Echo Secrets

Even masked, it's a bad habit

🔒

Use add-mask

Manually mask derived values

🔄

Rotate Immediately

If in doubt, rotate the secret

👀

Review PR Workflows

Be extra careful with fork PRs


Have you ever had a close call with secrets? You're not alone. Learn from my mistakes!

Share this story