acme/payments-api

What happened

Rollout blocked by memory limits, not by your code

The image built and the manifests applied cleanly. During verification, 3 of 4 pods were OOMKilled about 10 seconds after start and fell into CrashLoopBackOff, so the rollout never reached the minimum of 3 available replicas. The new nightly batch reconciler loads settlement batches into memory at boot, which pushes steady-state usage to roughly 249Mi against a 256Mi container limit that has not changed since March. Traffic was never shifted — the previous revision (2e7f83b) is still serving production.

Suggested fixes

1. 1

   Raise the memory limit to 512Mi

   The reconciler needs headroom above the 249Mi observed peak. Doubling the limit leaves ~50% margin and matches what identity-service runs with.

   bash

   Copy

   kubectl set resources deployment/payments-api -n payments-prod --limits=memory=512Mi --requests=memory=384Mi

2. 2

   Stream settlement batches instead of loading them at boot

   SettlementReconciler.loadPendingBatches() reads the full result set into a list. Paging it at 500 rows would keep the container under 300Mi without changing the limit.

3. 3

   Roll back while you decide

   Production is already on the previous revision, but rolling back explicitly clears the CrashLoopBackOff pods and stops the restart noise in alerts.

   bash

   Copy

   kubectl rollout undo deployment/payments-api -n payments-prod