(Reopened) Partial Outage across multiple systems

Major incident Web Application EU data center
2026-06-22 18:49 CEST · 3 hours, 11 minutes

Updates

Post-mortem

Incident Summary

On 22 June 2026, between 16:45 and 22:01 CEST, the PerfectGym Next platform in EU experienced a partial outage affecting the Web Application. The issue first impacted Integration activation and Aggregator Member related functionality and later caused broader degradation across the Web Application.

The incident was caused by a transaction-handling issue under elevated request pressure, which led to database connection contention across the platform. This resulted in slow responses, intermittent failures, and periods of severe degradation until containment and recovery measures were completed.

Impact on Customers

During the incident, customers may have experienced:

  • Slow or unresponsive behaviour in the Web Application
  • Intermittent errors when loading pages or submitting requests
  • Temporary unavailability of Integration activation and Aggregator Member related functionality
  • Temporary feature restrictions applied to protect overall platform stability

Studio devices and machines were not affected and remained operational during the time of the incident.

The incident affected customers in the EU region. During containment, the affected functionality remained restricted overnight to protect the overall Web Application while the system was stabilized and monitored.

Our Response

Our response followed the incident timeline:

16:54 CEST — Degradation detected

Monitoring identified rising error rates and increased response times in the Web Application. Investigation began immediately.

17:07 CEST — Incident declared

The incident was formally declared, coordination started, and mitigation planning began.

17:20–17:35 CEST — Initial containment measures applied

The team introduced targeted restrictions to the affected functionality in order to reduce pressure on the broader platform and protect overall availability.

17:39–18:06 CEST — Stabilization attempts under way

Additional recovery steps were taken to replace degraded application instances and adjust platform settings so the service could recover more safely under load.

18:30–20:10 CEST — Root technical pressure identified and reduced

The team identified database connection contention and locking behaviour as a major factor in the ongoing degradation. Operational measures were applied to reduce the blocking condition and throughput began to recover.

21:15–21:45 CEST — Final recovery steps completed

Remaining rollout and infrastructure blockers were cleared and configuration adjustments were completed.

22:01 CEST — Incident resolved

The platform recovered to a stable state and the incident was closed. The temporarily restricted functionality remained disabled overnight as a precaution and was restored the following morning after additional monitoring.

Resolution

The incident was resolved through a staged approach:

Containment — We temporarily restricted the affected functionality to reduce triggering conditions and protect platform-wide availability.

Remediation — We applied configuration and operational changes to reduce database connection contention, remove blocking conditions, and stabilize the platform.

Verification — We monitored platform recovery, kept the affected functionality restricted overnight as a safety measure, and restored it only after confirming improved stability the following morning.

Lessons Learned

Recovery controls must isolate affected functionality quickly.

Targeted feature restrictions were effective in reducing pressure on the broader platform and remain an important containment measure for platform-wide stability.

Connection contention needs stronger protective controls.

The incident showed that elevated request pressure in one feature area can create broader platform degradation when supporting resources become constrained. Additional protection and tuning are required to reduce this risk.

Opportunity for Improvement

Streamlining mid-incident operational adjustments. We can elevate our incident response capabilities by making critical runtime settings more dynamic. This optimization will allow the team to pivot faster and maintain peak efficiency during high-pressure scenarios.

Monitoring and recovery procedures will be improved further.

Follow-up work has been identified to improve early detection, reduce recovery time, and lower the chance of similar platform-wide degradation in the future.

June 24, 2026 · 11:05 CEST
Resolved

Incident Summary

On 22 June 2026, between 16:45 and 22:01 CEST, the PerfectGym Next platform in EU experienced a partial outage affecting the Web Application. The issue first impacted Integration activation and Aggregator Member related functionality and later caused broader degradation across the Web Application.

The incident was caused by a transaction-handling issue under elevated request pressure, which led to database connection contention across the platform. This resulted in slow responses, intermittent failures, and periods of severe degradation until containment and recovery measures were completed.

Impact on Customers

During the incident, customers may have experienced:

  • Slow or unresponsive behaviour in the Web Application
  • Intermittent errors when loading pages or submitting requests
  • Temporary unavailability of Integration activation and Aggregator Member related functionality
  • Temporary feature restrictions applied to protect overall platform stability

Studio devices and machines were not affected and remained operational during the time of the incident.

The incident affected customers in the EU region. During containment, the affected functionality remained restricted overnight to protect the overall Web Application while the system was stabilized and monitored.

Our Response

Our response followed the incident timeline:

16:54 CEST — Degradation detected

Monitoring identified rising error rates and increased response times in the Web Application. Investigation began immediately.

17:07 CEST — Incident declared

The incident was formally declared, coordination started, and mitigation planning began.

17:20–17:35 CEST — Initial containment measures applied

The team introduced targeted restrictions to the affected functionality in order to reduce pressure on the broader platform and protect overall availability.

17:39–18:06 CEST — Stabilization attempts under way

Additional recovery steps were taken to replace degraded application instances and adjust platform settings so the service could recover more safely under load.

18:30–20:10 CEST — Root technical pressure identified and reduced

The team identified database connection contention and locking behaviour as a major factor in the ongoing degradation. Operational measures were applied to reduce the blocking condition and throughput began to recover.

21:15–21:45 CEST — Final recovery steps completed

Remaining rollout and infrastructure blockers were cleared and configuration adjustments were completed.

22:01 CEST — Incident resolved

The platform recovered to a stable state and the incident was closed. The temporarily restricted functionality remained disabled overnight as a precaution and was restored the following morning after additional monitoring.

Resolution

The incident was resolved through a staged approach:

Containment — We temporarily restricted the affected functionality to reduce triggering conditions and protect platform-wide availability.

Remediation — We applied configuration and operational changes to reduce database connection contention, remove blocking conditions, and stabilize the platform.

Verification — We monitored platform recovery, kept the affected functionality restricted overnight as a safety measure, and restored it only after confirming improved stability the following morning.

Lessons Learned

Recovery controls must isolate affected functionality quickly.

Targeted feature restrictions were effective in reducing pressure on the broader platform and remain an important containment measure for platform-wide stability.

Connection contention needs stronger protective controls.

The incident showed that elevated request pressure in one feature area can create broader platform degradation when supporting resources become constrained. Additional protection and tuning are required to reduce this risk.

Opportunity for Improvement

Streamlining mid-incident operational adjustments. We can elevate our incident response capabilities by making critical runtime settings more dynamic. This optimization will allow the team to pivot faster and maintain peak efficiency during high-pressure scenarios.

Monitoring and recovery procedures will be improved further.

Follow-up work has been identified to improve early detection, reduce recovery time, and lower the chance of similar platform-wide degradation in the future.

June 22, 2026 · 22:01 CEST
Monitoring

The application is available again. We have further narrowed down the root cause of the issue and are currently monitoring the impact of the measures that have been implemented.

June 22, 2026 · 21:45 CEST
Update

Unfortunately, the measures taken so far have not been able to resolve the underlying issue. The application is experiencing widespread disruptions again. We are working with the highest priority to resolve the problem as quickly as possible.

June 22, 2026 · 21:19 CEST
Update

The application is available again. We are currently verifying whether the measures taken have resolved the underlying issue.

June 22, 2026 · 20:47 CEST
Update

The team added additional mitigations, which brought temporarily releave but did not fully resolve the issue yet.

June 22, 2026 · 20:05 CEST
Update

The team is checking options to deploy an earloer version of the service. This requires some additional preperations.

June 22, 2026 · 19:57 CEST
Investigating

The team is still imvestigating the issue.

June 22, 2026 · 19:29 CEST
Investigating

We are currently experiencing performance degradation and intermittent availability across several systems. Some features may be slow or unavailable. Our team is investigating the root cause.

June 22, 2026 · 18:49 CEST
Issue

It became apparent that the measures implemented following the previous outage did not resolve the underlying issue.

June 22, 2026 · 18:05 CEST

← Back