On March 18 between 15:41 and 17:21 UTC most Server Density service web checks failed to be scheduled to run.

This was caused by a code change in how the web checks component retrieves and dispatches service web checks to our worldwide testing locations. This component had a built in caching mechanism to allow faster retrieval of the tests to be executed and it was this mechanism that failed during this incident.

Following this incident, we have improved the response times and throughput capacity of the other Server Density components, that were the initial cause for that caching mechanism to be in place. This has allowed us to simplify the component by removing that caching mechanism altogether, making sure this issue would not happen again on account of a failed cache.