All issues and their side effects have been resolved.
Posted May 15, 2024 - 13:54 CEST
Update
We have gotten reports that the rolled up statistics data (elastic) is incorrect for some accounts for March. We are investigating, but can confirm it is off for those reported accounts. The source data is however correct, so we will need to re-roll it up.
Posted Apr 03, 2024 - 08:47 CEST
Update
Maintenance scheduled for rebooting the private network switch.
Posted Mar 25, 2024 - 14:25 CET
Update
The workarounds are holding up great. No issues reported today (Mar 20th). We are in the process of scheduling a maintenance window to update the firmware of the affected private network switch to see if that resolves the root cause. We have another (configured and tested) switch standing by in case the firmware upgrade does not help.
Posted Mar 20, 2024 - 15:26 CET
Update
We are continuing to monitor for any further issues.
Posted Mar 19, 2024 - 23:12 CET
Monitoring
We have reduced the load on the impacted connection and this has lead to a restore of the automatic VOD asset processing processes. We will keep monitoring these systems and the impacted connection for stability.
Posted Mar 19, 2024 - 20:06 CET
Update
We have found the component where the error occurs, we are in contact with several parties to see how to resolve the issue. The root cause was an software update to the switches of our network provider. Since that moment one of our private network switches is experiencing a very high load, while the same switch on the side of the "bridge" is not. When the load on that particular switch reaches 100% it starts dropping packages, causing delays and retries of uploading assets in our platform. We have diverted most of the use of this connection to lighten the load on that switch.
When we have more information we will update the statuspage.
Posted Mar 19, 2024 - 18:48 CET
Identified
The problem we had yesterday (Mar 18th) has not been fully resolved. Our network provider is currently working in the datacenter to resolve the problems.
Posted Mar 19, 2024 - 15:10 CET
This incident affected: Ingest (VOD Objectstorage and FTP).