Microsoft Cloud Services Stumble After Outage Hits Texas Datacenter

By: Kurt Mackie

Various Office 365 and Azure cloud services went down this week after a Sept. 4 lightning strike temporarily disabled Microsoft’s datacenters in the South Central U.S. region.

The lightning strike on a San Antonio, Texas datacenter temporarily knocked out the power, leading to service outages that have affected many regional South Central U.S. Azure and Office 365 services, as well as nonregional Azure Active Directory services. Power to the affected datacenter has since been restored, but Microsoft still listed multiple Azure services that are out for customers in that region, according to its Azure Status page.

The DownDetector.com site, which tracks service outages, offered this heat map, showing that Azure service outages appear to be mostly located in the Texas region:

Heat map from DownDetector.com showing Microsoft Azure service outage on the afternoon of Sept. 4, 2018.

Office 365 services were also affected for customers using hosting services from Microsoft’s San Antonio datacenter. The kinds of problems Office 365 users might encounter were listed as follows, according to this Office 365 announcement:

  • Exchange — Some users may be unable to access Outlook on the web. Email access through other protocols may be unaffected.
  • Power BI — Users may receive a ‘Server unavailable’ error or may be unable to log in.
  • SharePoint — Most impact has been mitigated, but a subset of users may be unable to make or save changes and may experience stale search results. Further details on SharePoint mitigation can be found under service incident SP147560.
  • Microsoft Teams — Users may be unable to access Office documents within Teams.
  • Intune — Affected users may be unable access the Intune portal or other functionality.
  • Education — Affected users may be seeing issues with services including School Data Sync.

The start time for the problems was listed as occurring at 9:09 a.m. UTC, which is about 2:09 a.m. PST.

The outage has affected developers using Microsoft’s Visual Studio Team Services, causing them to be unable to access their accounts, and reporting dashboards won’t load, according to a VSTS announcement. That announcement also specifically mentioned that the underlying issue was “a datacenter cooling outage in South Central US which is impacting all services in a specific part of the datacenter.”

The outage also affected users of Azure Active Directory, Microsoft Dynamics Finance and Operations and Lifecycle Services, according to a Microsoft Dynamics announcement.

So far, there are no workarounds for affected customers. At press time, Microsoft had not indicated when there might be a resolution.

Many of Microsoft’s services come with service-level agreements, typically assuring 99.9 percent uptime availability. Organizations can get a service credit if they apply for the downtime. However, these customers have to have had no other service options to claim the credit.

We track every new Microsoft Dynamics 365 and NetSuite job posting across North America, every single day, and October gave us the clearest read yet on where this market is actually heading. The short version is that ERP hiring cooled about 11% from September. That headline is true, and it is also misleading. Almost the entire drop sits in the Dynamics implementation tracks, while NetSuite held flat and demand for senior, in-house talent stayed strong. Whether this market feels soft or busy right now depends entirely on which role you are trying to fill. I pulled the full track by track breakdown into this month’s DynamicsFocus Recruiting Report, and it is attached below. If you have a role that has been sitting open since the summer, I would genuinely love to hear what you have tried so far and am more than happy to schedule a 30 minute Teams call to share potential strategies to rectify the situation, no strings attached. #MicrosoftDynamics #D365 #Dynamics365 #NetSuite #ERP #NetSuiteRecruiting #MicrosoftDynamicsRecruiting #TalentMarket #Hiring #AIAgents

Read More »