Customers of Microsoft Dynamics 365 Finance and Operations Enterprise edition were vulnerable to “man-in-the-middle” attacks for three months before Microsoft resolved the exposure. And according to the developer who discovered it, Microsoft only deployed a fix after increasing pressure that made the issue public.
German software developer Matthias Gliwka describes himself as a “curious world traveler, hacker and software engineer.” Details are hard to nail down on Gliwka, but he appears to be an independent, who maintains an infrequently updated blog. It was there, last week, where he described the 100-day journey to resolution of the Dynamics 365 cloud security issue.
Gliwka described in an August 17, 2017 email to the Microsoft Security Response Center (MSRC) (secure@microsoft.com), that he had noticed a vulnerability in the Azure-hosted D365FOE environments. As he described:
Each separate customer environment…uses the same wildcard server certificate (including the private key) for the domain *.sandbox.operations.dynamics.com, meaning the service hosted for Acme Inc. at acme.sandbox.operations.dynamics.com uses the same TLS wildcard certificate as Evil Inc. hosted at evil.sandbox.operations.dynamics.com.
Thus, one valid TLS certificate would serve them all, and Gliwka found it easy to access. And as he points out, sandbox environments are usually mirror instances of production instances (e.g., critical customer data in CRM); these used a wildcard certificate as well, for [customer].operations.dynamics.com.
The implications as Gliwka describes them:
- An attacker would be able listen and/or interject himself between the connection from the user to the server, impersonate the server, and so read all communication in clear text
- The attacker could modify the communication (e.g., by inserting malicious content)
- Because the attacker would use the original TLS certificate, the client receives no warning or error message
The solution, simply, is that each customer environment requires an individual certificate – but Microsoft seemed in no hurry. After five days, MSRC wrote back asking for details on real-life scenarios, but advised that “this typically would not meet the bar for security servicing.”
After several weeks of back-and-forth with MSRC and PKI Operations (Public Key Infrastructure) which Gliwka says went nowhere – even a very public tweet by Gliwka on October 4 – there was still no resolution.
Then German IT journalist Hanno Böck weighed in, and on November 30, opened a ticket on the Mozilla bug tracker. There he informed Mozilla:
According to the Baseline Requirements certificates with compromised keys shall be revoked within 24 hours. According to Gliwka’s description he has been in contact with Microsoft since August. In first replies Microsoft denied that this is a security problem, later they were unable to find the case again and later attempts by Gliwka were unanswered. The certificate is still valid.
I intend to make this public soon, but I wanted to make sure Mozilla is aware, as this obviously affects the root program. I’ll CC some people who commonly deal with CA issues.
The short story: within a few days, both wildcard certificates were revoked. “Future instances of Dynamics 365 will get an individual certificate with an individual private key,” advised Böck. Microsoft denied that the issue exposed production instances, where there is no remote desktop access. But Böck describes how a clever attacker could forward a customer from one instance to another, “as the certificate is valid for all instances.”