PerfectGym Next Status
Disruption | voucher loading
Updates
Incident Summary
On 5 October 2026, some of our clients experienced delays in voucher processing and failures affecting some payment-session requests through the PerfectGym Next Open API.
Our team began investigating immediately after receiving a report at 15:17 CEST. We identified two separate issues: a long-running database query that slowed voucher processing, and existing software defects that caused some payment-session requests to fail when customer-account creation requests overlapped.
The database query was stopped, restoring voucher performance, and the incident was closed at 17:04 CEST. Permanent fixes for the separately identified payment-session defects are currently in review.
Increased traffic during the marketing campaign amplified the number of payment-session failures, but did not introduce the underlying defects.
Impact on Customers
Affected customers could experience:
- Delays when applying or checking vouchers, with some measured voucher lookups taking very long
-
Errors preventing some payment sessions from being created successfully.
The response ran from 15:17 to 17:04 CEST, a period of 1 hour 47 minutes. This represents the time from notification to incident closure.
What Caused the Disruption?
Voucher-processing delays
An internal database query remained active for more than three days. Although it did not modify data, it prevented routine database cleanup from completing, making voucher lookups significantly slower.
The investigation initially gave this query insufficient attention because it appeared to be legitimate internal activity.
Payment-session failures
Separately, two services did not safely handle overlapping requests to create the same customer-account records. When these requests conflicted, some payment sessions failed instead of completing successfully.
Why monitoring did not alert us earlier
Voucher processing was delayed by queries waiting on internal database operations, while CPU usage remained normal. Our monitoring did not provide an actionable notification for this condition before our client contacted us. Investigation began immediately following the report at 15:17 CEST.
Our Response
| Time | Response |
|---|---|
| 15:17 | the report of increased errors reached our team, and investigation began immediately. |
| 15:17 – approximately 15:47 | The team assessed the scope, investigated potential causes, and coordinated communication. |
| 15:35 | Errors temporarily decreased. Investigation and monitoring continued to establish the cause and determine whether the improvement was sustained. |
| Approximately 16:00 | Payment-session defects were identified. Because these did not explain all reported symptoms, the team continued investigating. |
| 16:40 | The long-running database query was identified as the cause of voucher-processing degradation. |
| 17:04 | Voucher performance recovered, and the incident was closed. The payment-session fixes remained a separate follow-up item among others. |
Resolution
Stopping the long-running database query removed the condition affecting voucher processing and restored performance.
This action did not correct the separate payment-session defects. Fixes for those defects are currently in review and will be verified against overlapping and repeated requests before being considered complete.
Lessons Learned and Preventive Actions
- Improved monitoring: We improved our monitoring for long-running transactions within individual systems. We will also adjust monitoring to detect this type of database degradation even when CPU usage remains normal, so that these cases trigger an actionable notification to our on-call team.
- Stronger scrutiny of database queries: We are strengthening investigation guidance so that internal and read-only queries are assessed by their duration and production impact.
- More resilient payment-session creation: Fixes are in review to safely handle conflicting customer-account creation requests.
- Preventive runtime limits: We are evaluating automatic termination of excessively long-running transactions, with safeguards for legitimate operations and already made first adjustments.
- More complete incident investigation: We are improving our investigation tooling and guidance to ensure that finding one cause does not end the investigation while other symptoms remain unexplained.
The incident ‘Disruption | voucher loading’, which occurred between 2026-10-05 15:17 CEST and 2026-09-15 17:04 CEST, has been resolved.
Our engineering team will review the issue and implement additional measures to prevent similar incidents in the future.
If you continue to experience any problems, please open a ticket with our support team.
We apologize for any inconvenience caused.
We found the root cause of the incident and are applying counter measures at the moment
We are currently observing a high error rate in some of our API endpoints. Our team is working hard to resolve this issue. We apologize for any inconveniences and appreciate your patience during this time.
← Back