Database performance issues causing partial service outages across the platform

Incident Report for Hive

Resolved

This incident has been resolved.
Posted Oct 13, 2021 - 16:20 EDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Oct 13, 2021 - 15:13 EDT

Update

We have completed a manual scaling increase and are currently redeploying our services to re-sync and restore service.
Posted Oct 13, 2021 - 13:37 EDT

Update

We continuing to work on mitigating efforts while we work with our DB provider to restore our scaling functionality.
Posted Oct 13, 2021 - 13:03 EDT

Update

Our DB PaaS provider is currently experiencing an outage that has been preventing our normal scale-up process (https://status.cloud.mongodb.com/incidents/bkzhxk9db0nr) and we are continuing to work on other mitigating options while we wait for them to fully restore service.
Posted Oct 13, 2021 - 12:26 EDT

Update

We have identified the likely source and are currently working to remediate and restore service.
Posted Oct 13, 2021 - 10:31 EDT

Update

We are continuing to investigate this issue.
Posted Oct 13, 2021 - 10:29 EDT

Update

We are continuing to investigate this issue.
Posted Oct 13, 2021 - 10:28 EDT

Investigating

We are currently investigating this issue.
Posted Oct 13, 2021 - 10:28 EDT
This incident affected: Feature Services (Document Proofing, Messaging and Notifications) and Client Applications (Desktop Application, Mobile Application).