Resolved -
All environments are restored to pre-event status. Thank you for your patience and understanding.
Jul 20, 22:54 UTC
Update -
BlackLine GCP Production and Sandbox environments for both US and EU have been restored.
The BlackLine US2 and AMS Production environments are now fully operational.
The BlackLine US2 Sandbox environment is also fully restored. There may still be isolated instances of functionality that are not operational for the EU Sandbox.
All core systems should now be available. We are working to ensure full redundancy across all services.
Jul 20, 20:08 UTC
Update -
BlackLine GCP Production and Sandbox environments for both US and EU have been restored.
The BlackLine US2 and AMS Production environments are now fully operational.
The BlackLine US2 Sandbox environment is 99% restored. There may be isolated instances of functionality that is not operational. We are working to complete these restorations.
All core systems should now be available. We are working to ensure full redundancy across all services.
Jul 20, 18:01 UTC
Update -
We are continuing to work on a fix for this issue.
Jul 20, 17:37 UTC
Update -
BlackLine GCP Production and Sandbox environments for both US and EU have been restored.
The BlackLine US2 and AMS Production environments are now fully operational.
The BlackLine US2 Sandbox environment is 99% restored. There may be isolated instances of functionality that is not operational. We are working to complete these restorations.
Jul 20, 17:29 UTC
Update -
Saturday Morning Update:
The BlackLine US2 and AMS Production environments are now fully operational. There are still very isolated pieces of functionality that may not be fully operational, and we are working to resolve those issues.
GCP Environments are 95% operational. Users should be able to log in and utilize the majority of functions. Sandbox environments are 90% or more, and we are making progress this morning.
Jul 20, 15:00 UTC
Update -
The BlackLine US2 and AMS Production environments are now fully operational.
The BlackLine US2 Sandbox environment is 95% restored. However, some services are still non-operational, and we are working to make them operational.
We are actively working through the GCP environments; however, we do not have a completion estimate.
Jul 20, 02:24 UTC
Update -
The BlackLine US2 Production environment is now fully operational.
The BlackLine US2 Sandbox environment is 95% restored. However, some services are still non-operational, and we are working to make them operational.
AMS environments are approximately 80% restored, with some services impacted.
We are actively working through the GCP environments; however, we do not have a completion estimate.
Jul 19, 23:40 UTC
Update -
BlackLine’s US2 environments are now at 95% restored. However, some services, such as Account Analysis, Excel Imports, and Journal services, are still non-operational. We are working to get those services operational.
AMS environments are approximately 80% restored, with some services impacted.
We are actively working through the GCP environments; however, we do not have a completion estimate.
Jul 19, 21:50 UTC
Update -
BlackLine’s US2 environments are about 90% restored. However, some services, such as Account Analysis, Excel Imports, and Journal services, are still non-operational. We are working to get those services operational.
AMS environments are approximately 80% restored, with some services impacted.
We are actively working through the GCP environments; however, we do not have a completion estimate.
Jul 19, 19:59 UTC
Update -
BlackLine is utilizing all available resources to restore services to our customers in the wake of the global issue that surfaced last night. While the root cause has been fully understood, we appreciate your patience as BlackLine personnel must manually confirm each affected server, which is a complicated effort. Please be aware that there is no issue with BlackLine software; all data is fully secure, and we are working at top priority to provide complete functionality as quickly and safely as possible.
On a related note, please be advised that all scheduled BlackLine release and update activities for this weekend have been postponed.
Jul 19, 17:45 UTC
Update -
US2 PROD+SB clients may be able to log in however not all services are online and can affect imports, job processes, reporting, etc. BlackLine engineers are currently triaging and working to restore service.
Jul 19, 12:38 UTC
Update -
We are continuing to work on a fix for this issue.
Jul 19, 11:53 UTC
Update -
BlackLine engineers are currently triaging and working to restore service.
Jul 19, 08:20 UTC
Update -
BlackLine has identified an issue with our anti-malware software impacting Windows systems. The team is currently triaging and working to restore service.
Jul 19, 07:33 UTC
Identified -
BlackLine has identified an issue with our anti-malware software impacting Windows systems. The team is currently triaging and working to restore service.
Jul 19, 06:55 UTC
Update -
Due to a larger outage across the internet some customers may experience issues with connectivity to BlackLine. We are monitoring and will continue to work with providers to address connectivity.
Jul 19, 06:28 UTC
Investigating -
Customers may be experiencing an issue where BlackLine Production instances are unavailable. BlackLine teams are investigating the issue. We will provide regular updates on this page until the issue is resolved.
Jul 19, 06:06 UTC