System Incident: November 15, 2022

Subject: iVision System Incident - Drop & Go

At around 5:10am PST, several system users reported experiencing delays with Drop & Go file delivery and other errors with specific API endpoints.

IMPACT: Approximately 3,000 drop and go files did not process between 12am and 6:33am PST. In addition, specific API endpoints would return "SENT PENDING" status response for outbound messages in the queue.

ROOT CAUSE: Engineers determined that a virtual server responsible for the FTP server, Drop & Go file handling, and Text Reminder Workstations had unexpectedly logged out and didn't recover automatically.

ACTION: Engineers started investigating the incident at 6:00am PST and were able to identify the cause of the issue and resync the affected system server. The Drop & Go files started processing at 6:33am PST and were completely caught up by 7:35am PST. The resync of the affected server also resolved the "SENT PENDING" status on queued messages.

Over the coming days, our team will be working to implement additional safeguards and backups to prevent this type of incident in the future.

If you have any questions or concerns, please reach out to our Support Team by calling us or texting us at 866-655-5302 or by emailing support@ivisionmobile.com.