This incident was caused by two factors. The first factor was a spike in document uploads that caused the OCR process to time out for 1,875 documents. The second factor was a bug that prevented these documents from moving onto the next verification step following the OCR time out. Our engineers are currently working on the bug fix, which will be deployed in the coming weeks. We are also adding additional monitoring around OCR time out events to catch potential issues sooner to reduce the chances of affected documents becoming overdue.
Posted Sep 11, 2023 - 17:24 EDT
Resolved
We have finished processing all the documents that were in the backlog, and our human-in-the-loop document verification system is functioning normally. We are still investigating the root cause.
Posted Sep 06, 2023 - 18:57 EDT
Investigating
Since approximately 2:30pm ET, there has been a backlog of documents waiting to be processed in our human-in-the-loop document verification system. We are restarting these documents and working through the backlog. We are still investigating the root cause.