We have confirmed the bad deploy to our UI production servers resulted in this problem. It was not an issue with the codebase itself, rather the branch not correctly propagating to our web servers.
Our team are continuing to look into why the deploy failed and we're implementing additional measures to ensure it does not happen again.
Posted Nov 19, 2015 - 15:57 GMT
Identified
At 13:36 UTC we deployed code to our UI which on initial checks did not deploy to all of our web servers correctly. We reverted to a previous branch at 14:02 UTC which propagated correctly and resolved the UI errors.
Payload processing, alert delivery and the API was not affected, this was isolated to our UI only.
We are continuing to look into this and will share updates as we have them.
Posted Nov 19, 2015 - 14:16 GMT
Investigating
We're looking into sporadic problems with our UI which is responding with 403 or 500 errors. On initial investigation payload processing and alert delivery is not affected.