B (172:34)
Okay, so just to be clear, MongoDB itself probably never needs to be publicly exposed. It would normally be sitting behind a publicly exposed web app server of some kind, serving as that web app server's back end database. MongoDB itself really doesn't have any public exposure use cases. We've been talking a lot recently about the need to make these sorts of public exposure mistakes far, far more difficult to make. When I was swooning over Cisco's promises a month or two back, it was because the noises Cisco was then making strongly suggested that this might have finally sunk in. We can only hope and pray anyway. So we connect with TCP to the server's port 27017. Mongo uses a binary variant of JSON J S O n called bson B S o n. You know, binary object notation. So the requests that sent the request that's sent to the Mongo server contains one of these Beeson messages, and for the sake of the speed of transmission, that request can optionally be compressed using Z libe. Compression makes the message smaller, of course. So one of the 32 bit values in the requests header at the front of the message, which specifies that this request has been compressed, indicates the original un uncompressed, you know, the decompressed size that the message would be, the way that what it originally would be, and what it would again be when decompressed by the receiving server. So this allows the receiving MongoDB server to request the allocation of a block of memory from the underlying actually it's the runtime, the C runtime. We'll get to that in a second into which MongoDB will decompress the message. So an attacker creates and sends a server request which claims to contain far more data than it actually does. In response, the server allocates the requested memory. An attacker might claim, for example, that the uncompressed request will require one memory million bytes, one megabyte, when in fact it only needs 1K. The critical flaw is that once MongoDB has finished decompressing, it never checks the actual resulting size of the newly decompressed payload. It trusts the data the user provided, using that as the actual size of the payload. Now, I need to stop here to hover over that phrase a bit longer. That phrase being it trusts the data the user provided. If we were to produce a list of the root causes behind many of the worst flaws that we that that have been found in software, trusting user provided input would definitely be right up there near the top, if not perhaps in first place, since even buffer overflows typically result from the similar mistake of trusting and using something that a malicious user deliberately provided. In this case, we have a deliberate buffer underflow that results entirely from from trusting input from the user. Okay, so what's the big deal about allocating an oversized buffer that's not needed? In many contemporary languages, memory allocated from a program is cleared to zeros. It's initialized to zeros before it's returned for use by the caller who requested an allocation of memory. But Malloc, the memory allocation function used by C and C does not bother doing so. This is part of the trusting, performance oriented but dangerous legacy of C. Since zeroing RAM takes time and blows the processor's cache, C deliberately returns uninitialized memory. And wouldn't you know, MongoDB is written in C. The result of the bug is that multiple megabytes of the server's raw internal data can be exfiltrated to the attacker. This data might and often does contain clear text passwords and credentials, session tokens, API keys, customer data, database configurations, system inf info, docker paths and client IP addresses, and so on. In short, all of the internal operations of the server that would otherwise never be made available to anyone whether they had authenticated and were legitimate user or not. So to sum this up, an attacker sends an otherwise valid MongoDB message which indicates that it employs compression, but that compressed message is deliberately manipulated to specify a hugely exaggerated claim about the message's uncompressed size. Since the server has no way to know in advance, MongoDB obtains a large and uninitialized buffer from the C runtime based upon the attacker's messages claimed need. MongoDB's built in Z Live decompresses the much smaller compressed data into just the front of the huge decompression buffer, thus avoiding overriding the mother load of data that's already sitting there in that buffer. Subsequent commands then instruct the database server to return to this attacker what it believes is the user's provided data, even though it's actually megabytes of whatever data had been previously used and left behind by previous database operations and internal workings of any kind. It's obvious now why this critical flaw was named Mongo Bleed, right? And also why it was given a CVSS of 8.7. Although it doesn't allow a remote attacker to execute their own code on the server, it's a data exfiltration flaw of the highest order. That's just about as bad as it gets. Proof of concept code has been published on GitHub, and the flaw is trivial to exploit. There's nothing like it only works less than one time in 1000, and only when your code wins some slippery internal race condition or something. No, this one is extremely straightforward. It obeys simple rules. The attacker receives much more than a tiny trickle of data, you know, over time without raising any alarms, without crashing the server or otherwise calling any attention to itself. The abuse of this long standing vulnerability that's been present in every version of MongoDB published in the last eight years allows remote bad guys to freely rummage around inside the more than 87,000 currently online and publicly exposed instances of MongoDB. They're able to keep sucking out and examining megabytes of a server's data that is assumed to be utterly private internal working data, and which might therefore and does, it turns out, often contain very juicy information. It's always fun to see Kevin Beaumont's take on these things. On December 16th, day after Christmas, Kevin posted Somebody from Elastic Security decided to post an exploit for CVE 2020514847 on Christmas Day. The vulnerable, which dropped just before Christmas in theory allowed memory read without authentication. Patches are available. It impacts every version of MongoDB going back about a decade. Another vendor decided it would be a great idea to post technical details on Christmas Eve and he has a link to an OX Security blog. He said the exploit dropped yesterday and is the first public exploit. It's dubbed Mongo Bleed. I validated that said exploit is real. You can just supply an IP address of a MongoDB instance and it'll start ferreting out in memory things such as database passwords which are plain text, AWS, secret keys, etc. The exploit specifically looks for those class of credentials and secrets as well. The Internet footprint of MongoDB is very large, over 200,000 instances. Because of how simple this is now to exploit, the bar is removed. Expect high likelihood of mass exploitation and related security incidents. The exploit author has provided no details on how to detect exploitation in logs via products like Elastic. Advice would be to keep calm and patch Internet facing assets. So now we know all about this mess. And Kevin's ending advice to keep calm and patch Internet facing assets reminded me of something Leo and I talked about long ago. We made the observation many times in fact, that once a user's system had been infected by something, anything, it was never really again possible to trust it. How could anyone ever know with 100% assurance that every last bit of an infection had been removed? And what about whether an infection might have spread over the local network work to infect other assets? In short, it's a real mess. We've also seen instances where huge problems resulted when companies did not take prior intrusions seriously enough. The advice. The advice is always to, you know, rotate all credentials which may have been may have had any chance of being exposed, meaning invalidate any long term authentication tokens, change all passwords, and so on. But as I said, we keep seeing instances where companies, for one reason or another, you know what? Oversight, laziness, lack of belief that it was really necessary? Who knows? But for whatever reason, they failed to adequately and fully remediate the consequences of a breach, only to suffer, again, often even worse. So now consider the plight of corporate users of publicly exposed MongoDB servers. You're told that for the past eight years, the database server you've been relying upon has contained a flaw that allows for effectively unfettered mass exfiltration of your server's internal working memory, which contains myriad private credentials, past database search results, and essentially any and all proprietary information to which that server may have had internal access, or may have been storing and retrieving over time. To call this a mess is truly an understatement, and this mess is now squarely in the laps of every enterprise that was using a publicly exposed MongoDB server. My question is, why was even a single instance of MongoDB publicly exposed? I'm sitting here right now as I talk to Leo and our audience in Southern California. From my location here, I have access to any and all of those 87,000 some instances of Mongo DB. Why? Why do I have access? Why can I send out a TCP sin packet to port 27017 of any to any of those 87,000 IPs and promptly receive a TCP SYNAC packet inviting me to complete the TCP handshake connection? Why? I have no need to ever do so. Whoever runs that MongoDB instance certainly doesn't want or expect me, sitting here in Southern California to be able to connect to their database server. But I can. Why? By now, I hope that everyone in this podcast's audience understands not only that this is wrong, but just how wrong it is. If I were to confront whomever it was who set up any given instance among those 87,000, that IT person would probably respond, well, we've password protected access to our database and you can't do anything without that. Oh yeah, Mongo bleed baby. No authentication needed. The decompression of the message is pre authentication and never requires any form of authentication for its exploitation. One of the refrains everyone listening to this podcast has been hearing from me, beginning last year when it finally so clearly crystallized after we all witnessed mistake after mistake after mistake, which all carried the same pattern. This pattern, which is that authentication does not work. Now, the world depends upon and turns on the strength of authentication. So I obviously don't mean that it can't work. What I mean is that it cannot be absolutely dependent upon to work. In my hypothetical conversation with that MongoDB IT person, their defense of their databases Utterly unnecessary public exposure was that I didn't know the secret handshake, so they didn't feel the need to take every possible precaution. The massive sweep of today's mongo bleed vulnerability is the direct consequence of that wrong way of thinking. That way of thinking is obviously defective and wrong. Sitting here in Southern California, I have no need to be able to connect to any of those 87,000 MongoDB servers, even if only to test the strength of their authentication. I should not be allowed to do that, but I can. And that's on them. That's on each and every one of them individually. This erroneous reliance upon remote authentication, which we keep seeing over and over, does not work. It's perhaps the single most important thing that has to change in today's Internet networked world. And what's most galling is that it's not about flaws or mistakes. Right? It's entirely about policy and caring. If we cared to, we could fix it.
