Major service outage

On July 18, 2025, Lokalise became unavailable at 12:31 UTC due to a maintenance error that disabled an encryption key needed for data access. We fully restored service by 13:32 UTC with no loss of customer data.

What happened?

During a periodic scheduled clean up procedure, an encryption key presumed to be unused was disabled. This key, in fact, was still in use by one of the auxiliary databases. With the key disabled, the application was unable to access data and this led to a disruption of service.

Our team identified the problem quickly and worked closely with our cloud provider’s support team to resolve the issue. The encryption key was re-enabled and a point-in-time recovery was performed to restore the service. The main application was back online by 13:25 UTC, and all services were fully operational by 13:32 UTC.

The root cause of the incident was that the encryption key was mistakingly referenced in two terraform state files. One of the state files was no longer associated with any infrastructure, the encryption key therefore was deemed unused, and scheduled for removal.

Impact

• 12:31 – 13:25 UTC: Lokalise application was unavailable.
• 13:25 – 13:32 UTC: Some background services were still recovering.

What we are doing to prevent this in the future

We take reliability very seriously and have performed a thorough analysis of this incident. We are going to make a number of improvements to prevent this type of incident from happening in the future. These include:

• improve procedure and automation around disabling and decommissioning of encryption keys to cross-check all data storages to ensure the key is not used anymore
• improve monitoring to detect the databases going into grace period when an encryption key is disabled, but the database is still functional; this will allow us to re-enable the key without the need in point-in-time recovery and service interruption
• review terraform state files for any keys and other resources referenced in multiple state files and address findings if any

We sincerely apologize for the disruption this caused. Thank you for your patience and continued trust in Lokalise. If you have any questions, feel free to contact us at support@lokalise.com.