Railway Security Breach: What Happened and What to Do

This story is being updated as new details emerge.

Railway is a popular cloud platform for deploying applications, used by thousands of startups and developers. The incident reaches beyond the platform itself: if keys to external services were stored in its environment variables, those services are at risk too. Here is what we know so far, what the company says, and what users should do.

📚 Table of contents

• 📌 What happened
• 📌 Incident timeline
• 📌 What Railway says
• 📌 Why deployments are paused
• 💼 How this affects your projects
• ✅ What Railway users should do
• 📌 What is still unknown
• ❓ Frequently asked questions (FAQ)

🎯 Section 1. What happened

According to Railway, the company detected anomalous activity in the systems storing customer environment variables on the morning of Oct. 21. A preliminary investigation showed that an unauthorized party may have accessed encrypted variables and, in some cases, their decrypted contents.

Developers typically store their most sensitive data in environment variables, including API keys, database passwords, and private container registry credentials. As a result, the incident affects not only the platform itself but also every external service whose keys were stored there.

🎯 Section 2. Incident timeline

• Oct. 21, morning — Railway detects anomalous activity in systems handling environment variables.
• Oct. 21, around 20:44 UTC — The company publishes a security disclosure on its blog and pauses all deployments.
• Oct. 21–22 — Users receive email notifications from Railway; AI providers, including OpenRouter, proactively disable keys and notify their owners.

🎯 Section 3. What Railway says

The company believes the attack is part of a series of recent attacks on platform companies aimed at obtaining keys to AI inference providers. Railway has notified these providers so they can proactively revoke keys.

Based on initial estimates, only a subset of users was affected. However, Railway recommends that all customers immediately rotate all environment variables, API keys, and private registry credentials, and check the related service logs for suspicious activity. The company said it will contact users whose accounts are confirmed to be affected directly.

🎯 Section 4. Why deployments are paused

While the investigation continues, any new deployment means the platform must read and decrypt secrets again, including new keys users have just entered. If the vulnerability is still open, those fresh keys could leak as well. So Railway is closing the hole first and only then resuming builds.

Services that are already running continue to operate. Only new deployments, restarts, and applying changed variables are blocked.

🎯 Section 5. How this affects your projects

If your application on Railway uses AI APIs, you are likely already seeing the following:

• 401 Unauthorized (or 402 Payment Required) errors from OpenRouter and other AI providers, despite a positive balance and unchanged code;
• the “Limited Access — Deploys have been paused temporarily” banner in the Railway dashboard when you try to redeploy;
• new values in Variables not taking effect: the container keeps running with the old variables until it restarts.

We ran into this firsthand: in one of our Spring Boot projects deployed on Railway, all requests to OpenRouter started failing around 01:30 UTC on Oct. 22:

401 Unauthorized on POST request for "https://openrouter.ai/api/v1/chat/completions": [no body]
org.springframework.web.client.HttpClientErrorException$Unauthorized

The account balance was positive, and the code had not changed. Shortly afterward, an email from OpenRouter arrived about the proactive revocation of the key. A new key worked locally right away, but it could not be rolled out to production: Variables were updated, yet the container kept running with the old, already revoked key. Later, the 401 turned into a 402, which also came from the old key rather than a billing issue.

In short, the key has already been revoked, and a new one cannot be deployed until Railway restores deployments. The problem is not in your code.

🎯 Section 6. What Railway users should do

1. Rotate all secrets, not just AI keys

Prioritize credentials that present financial or data loss risks:

1. Databases — PostgreSQL/MySQL passwords. If you use an external database (Neon, Supabase, etc.), change them immediately.
2. AI providers — OpenRouter, OpenAI, Anthropic, and others, including keys that have not been disabled yet.
3. Storage and media — S3, Cloudinary, R2.
4. Email services — SMTP, SendGrid, Resend.
5. Application secrets — JWT, session keys, OAuth client secrets.
6. Private registries — Docker Hub, GHCR.

2. Audit logs for unauthorized activity

• AI providers: check the Activity/Usage page for unfamiliar models, unusual usage spikes, or off-hours requests.
• Databases: look for connections from unknown IP addresses.
• Cloud storage: review logs for unexpected downloads or deletions.

3. Prepare new values in advance

Generate new keys now, store them in a password manager, and enter them under Variables. They will take effect with the first deployment once Railway lifts the pause. After service is restored, a manual redeploy will likely be required.

4. Watch out for phishing

Major security incidents often trigger fraudulent emails disguised as official notices "from Railway" or "from OpenRouter." Go to your dashboards directly instead of following links in emails.

Caveat: databases hosted on Railway

Changing a database password now will cause your application to lose its connection until the new password can be deployed, and deployments are currently blocked. Teams face a choice between short downtime and the risk of unauthorized database access. If the database contains user data, rotating immediately is the safer choice.

For a practical guide to storing secrets so that the next incident like this costs you as little time and money as possible, see "Secrets in SaaS: storage, rotation, and a breach response plan".

🎯 Section 7. What is still unknown

• The exact number of affected accounts.
• How the attacker gained access to the systems storing variables.
• When deployments will resume — Railway has not provided a timeline.

The company posts updates on status.railway.com.

❓ Frequently asked questions (FAQ)

My Railway service is still running. Does this affect me?

Running services stay online; only new deployments are blocked. However, if your secrets were stored in Railway variables, you should rotate them anyway. Railway recommends this to all users.

Why is my application suddenly getting a 401 error from OpenRouter?

Most likely, OpenRouter proactively disabled the key that was stored on Railway. Check the keys page in your OpenRouter account and your email. A disabled key cannot be reactivated; you need to create a new one.

Why did the 401 turn into a 402 Payment Required error?

If your OpenRouter balance is positive and your app is still running with the old key, the 402 is most likely related to the revoked key as well, not to billing. Test the new key locally: if it works, the error should go away once you deploy the new value.

I updated a variable on Railway, but nothing changed. Why?

Environment variables take effect only on deployment or container restart. While deployments are paused, your service keeps running the existing build with the old values.

Can I hardcode a new key directly in the source code?

No. Changing the code still requires a deployment to take effect. On top of that, committing keys to a repository is a security risk of its own: OpenRouter and other providers automatically revoke keys they detect on GitHub.

Source: Railway — Security Disclosure: Wednesday October 21st