Initial Architecture and Motivation
Initially, orders were automatically retrieved by our ERP Odoo. Odoo called the Magento API and fetched the orders. For greater fluidity, we decided to push orders ourselves: on the after save event of an order, if it was in an eligible state, we published the order into our own queue (RabbitMQ & custom queue).
The queues are processed by a supervisor on our back-office server. When reading the queue, we take the order and send it to Odoo (or, more precisely, to avoid having two different processes, we notify Odoo about the order so it can pop just that order).
Note: Odoo still fetches orders at regular intervals as a safety net in case of issues on our side.
Why the queue? We chose to use a queue because, logically, we avoid synchronous API calls as much as possible: the API can be slow or unavailable, and in such cases, the user placing the order would suffer. When you’re processing e-commerce orders—especially during high-volume periods—you don’t ask a buddy to wait while you call Odoo. Instead, you queue the work and process it asynchronously, keeping the user experience smooth and the storefront responsive[.
Everything worked perfectly in pre-production. We deployed to production.
The Bug: Order Status Rollback in Production
After a smooth pre-production phase, we deployed to production. The next day, we noticed that orders paid via Hipay were being incorrectly cancelled. Hipay’s cron job, which runs 24 hours after order placement (configurable delay), was cancelling orders that were not in “pending” status, but we found many orders in “new” and “pending” status that were actually paid and even invoiced.
Investigating the sales_order_status_history table, we saw status changes, but no trace of a return to “pending”, a clear anomaly. At first, we suspected a Hipay bug, but on closer inspection, we found many orders in state “new” and status “pending” that were paid with Hipay and even invoiced. Checking the sales_order_status_history table revealed the status changes for these orders, but no trace of a return to “pending” status—something was actively rolling back these statuses.
We acted immediately by modifying the Hipay cron to check if orders were invoiced before cancellation, preventing further damage. Analysis showed the first affected cases appeared one hour after deployment, strongly suggesting our custom sync was the culprit. We looked at the code for sending orders to Odoo, which seemed the most likely culprit, though we didn’t fully understand why. We commented out this code in production. Result: no more problems! (We had to wait before celebrating, as not all orders were affected, of course.)
Root Cause: Transaction Boundaries and Race Conditions
Root cause
When Hipay processes a payment, it triggers a webhook that updates the order, invoices it, and saves everything—all within a single database transaction. Our custom sync listened to the after_save event, publishing the order to the queue almost instantly. The queue consumer, running via supervisor, then reloaded and processed the order before the transaction was committed.
As a result, the consumer saw and saved an outdated order state, effectively rolling back the status to “new_pending” even after Hipay had finalized the payment and invoicing. The supervisor was too fast: the consumer loaded the order before its status change was committed! Hipay then finalized its save with the correct info, but the consumer, which had loaded an order in “new_pending”, came to resave the ‘send_to_odoo’ flag at the same time as the rest. Since we save the whole order and not just the field, Magento resaved the old state and status.
The fix
Instead of using the after_save event, listen to the after_commit event. The after_commit event is only triggered after the transaction is fully committed, ensuring that any consumer or background process will always see the latest, consistent state of the order. This prevents the race condition where a consumer could reload and resave an outdated order state.
Key Technical Lessons and Best Practices
- Event choice matters: In Magento 2 (and similar systems), always prefer
after_commitoverafter_savefor actions that depend on the final, committed state of an entity—especially when integrating with queues or external systems - Queue consumers and transactions: Near-synchronous queue processing can introduce subtle race conditions if consumers act before transactions complete. Design your architecture to respect transaction boundaries.
- Asynchronous processing is essential: Real-time synchronization between e-commerce and ERP systems requires careful handling of async flows. Queues (like RabbitMQ) are fundamental to prevent blocking user actions, but you must pair them with proper event handling
- Hybrid sync strategies improve resilience: Combining real-time queue-based updates with periodic polling (as Odoo does) creates a robust fallback mechanism, ensuring no orders are lost even if the primary sync fails
- Monitoring and timing detection: When debugging, check not just the code, but the timing and transaction context—especially after deployments. Anomalies appearing exactly one hour post-deployment are rarely coincidence.
- Payment processing and integration complexity: Payment modules like Hipay operate in multi-step, transactional contexts. Any custom sync must respect these boundaries to avoid corrupting payment state
Conclusion
This case highlights how a small architectural choice (event listener) can have major consequences in production. By switching from after_save to after_commit, we resolved a critical order status rollback issue in our Magento 2, Odoo, and Hipay integration. The initial motivation, avoiding synchronous API calls to keep the storefront responsive was sound; the implementation needed refinement. The experience underscores the importance of understanding transaction boundaries, event timing, and the need for robust monitoring in e-commerce platforms. Always test integration scenarios under realistic conditions, respect transaction semantics, and structure your content to clearly communicate both the problem and the solution, helping others avoid similar pitfalls in their own Magento and ERP integrations.