> It's fortunate that the steps required here aren't particularly complicated for the devices you can update, and the advice essentially boils down to 'update them'. The problem is the devices you can't update or update easily.
> Update everything that ships a CA bundle: your OS ca-certificates package, certifi, Node, Homebrew, and rebuild your container images once the base images pick up the change.
> Devices you can't update are the hard part. Routers, printers, TVs, appliances with frozen firmware will trust these certificates until expiry, which is July 2039...
Quick -- we need to MITM all the smart TVs so they can be jailbroken in one fell swoop!
My oven (which has a camera and WiFi, don't ask, it's a great oven otherwise and doesn't exist without WiFi) received a software update this morning. I can't help but wonder if it's related.
new email from wayne, who is the original reporter:
"I've had more than a few emails asking on how this was discovered. The simple answer is: https://github.com/google/paranoid_crypto
Yeah, out of the box. CCADB was intended to be a quick corpus test before looking at subscriber certificates...
I've done a far more thorough check of all existing CCADB certificates, thankfully no more hits. Most of my time has been spent going over the basics that everyone assumed someone else had already done - a critical mistake to ever make near security.
So unless someone has significantly more computational resources or access to more cryptanalysis research the low-hanging fruit has been caught so far."
https://groups.google.com/a/mozilla.org/g/dev-security-polic...
> https://github.com/google/paranoid_crypto
This repository was archived by the owner on Aug 19, 2026. It is now read-only.
I guess now that we've checked, nobody will ever make a mistake like this again so we don't need this anymore? :)
Archived means it’s still available but signals that it won’t be developed.
Last commit was 2 years ago anyway.
You can still use a read-only archive.
I have to be honest, that hasn't stopped me from trying!
In retrospect, having 100 trusted certificate authorities might have been excessive.
In retrospect, not having a tiny competency check may have been an oversight.
That really depends how you define "trusted certificate authorities".
If you're talking about root certs then it's almost 350 currently trusted by at least one major operator.
If you include intermediates, you're talking more than 3,600!
How are they trusted if they can just generate any key they like in their back room?
"Trusted" means how people treated them, it doesn't mean the extent to which they deserved trust.
[delayed]
That's the thing about trust -- in order to get into the root key store for windows, chrome, linux, etc they were trusted not to do that.
they violated that trust.
Correct, we only need zero.
Interesting, what would be the alternative?
I mean specifically for the use case of the browser warning the user if there is a MITM between them and e.g. their bank.
Stop validating names, start signing content.
Scan a QR code when you open the bank account, add it to the keys you trust as authorities on financial things. When your browser runs accross data tagged "financial" it checks the trusted keys in that domain and renders it only if it was signed by a key that is trusted in that domain (the domains prevent your bank from abusing that trust by causing you to see political ads or some such).
If you ever see something that you suspect is malicious, you untrust whichever key signed it and take it up with whoever asked you to trust their key. If you verify it's malicious, gossip the bad keys to your friends so they can distrust also.
That solves integrity but not confidentiality. It does not protect against MITM.
[delayed]
Make dns secure and put certs / public keys in dns. Control over dns already gives you the right to get tls certs. It should just be connected directly. We don’t need CAs.
He's right. DNSSEC is a mess, and we should fix that. We should fix that whether or not we delete CAs because CAs rely on DNS to verify that you own your domain. If DNS isn't secure, neither are CAs.
In the long run, fixing bugs is almost always better than working around them. DNSSEC is a bug. So are CAs. We can and should fix both.
I have seen their comments across this site for a very long time. Unfortunately I do not respect or put any stock in what they say, because of how unbelievably rude and dogmatic they are, and always have been.
"As a rule, strong feelings about issues do not emerge from deep understanding."
A secure DNS is valuable whether or not we get rid of CAs, since CAs rely on DNS queries to verify domain ownership.
And if we have a secure DNS, then CAs are completely redundant. I'd much rather trust a handful of domain registrars than a handful of domain registrars, plus 100 random questionably competent organisations all over the world. Especially when any one of them can generate keys for every domain on the planet.
Isn't this what DNSSEC solves?
> That still requires trust stores to be distributed to the clients
There's no such thing as a "trust store" in practice. To be "trusted" you just need to demonstrate you have control over a DNS TXT record, and we don't need a special "store" or intermediary for that.
Certificate authorities are an artifact of a bygone time and an entirely different model of internet infrastructure.
You just have one big one on the root.
(Half joking).
I quite like how i2p does it. Public keys are part of addresses, human readable names are user defined and shared but static.
For me the complexity of anything connected to public internet is now over what I can secure.
I genuinely can't say I secured a PC, when it needs to phone home for a configuration at this point.
We are most likely over the line for needing to airgap a personal PC.
The first time you go to the bank, you do the "trust exchange" (maybe scanning a QR? People sure love that) that verifies future connections, similar to how you acquire digital certificates today when you deal with the government online and similar.
Edit: > Every time you access your bank on different devices normalizes it so that visiting a typo domain would show the dialog and many would just click through.
Show the dialog? What dialog? There is no dialogues here, you cannot login to the bank unless you've verified the connection. You do it with your phone, then you can propagate the trust from there to other devices if needed.
> If we want to improve on this, it has to come with the current level of ease of use at a minimum.
OR it has to be a situation catastrophic enough. I don't think this one is, but it's hard to be optimistic for the future
Presumably you had some reason to trust the bank in the first place. Maybe a recommendation from somebody you know? Get the key from them.
The current level of ease is not being achieved securely. If it's not possible to do so, we should walk it back.
Every time you access your bank on different devices normalizes it so that visiting a typo domain would show the dialog and many would just click through.
Like...TOTP or FIDO2?
[dead]
or, perhaps you need more so that one being stupid doesn't effect so much.
Each CA is like a padlock locked in series with other padlocks. The weakest padlock is the only one that matters.
In a world with CT logs and CAA records, is that actually the case?
Yes, because as we are seeing with this incident, plenty of things don’t check CT logs (and thus can still be hacked if CT isn’t published for bad certs), and CAA records are fundamentally advisory.
> perhaps you need more so that one being stupid doesn't effect so much.
So you create more surface area for vulnerability? We might already be at or close to the sweet spot. Security is hard.
I would assume that the client would need to validate the certs against more than 1 trust path (for the comment to make sense)
I reckon that was an attempt to fend off the enemy. "Everyone expects us to set 196 bits of entropy, but you know what they won't expect? A single byte!"
Okay hear me out. People are complaining about unpatcheable IOT device and thats indeed concerning. But can’t we use the bad keys to generate seemingly value certificates that can be used to jailbreak these devices?
EDIT: seems its already happening? https://x.com/otomir23/status/2112697830687047918
Yeah this is absolutely possible, and while they are a few good opportunities like this, it also means that someone else can decrypt all of your traffic in the same way.
Yeah, it's happening in the wild already. Seen more cases like that.
I really hope someone manages to root the Google Home speakers. Been waiting for ages to be able to repurpose them for Home Assistant integration!
Quick question for the crypto experts. I thought that the goal of a true random number generator was that all values between 0 and MAXCOUNT are equally likely, and therefore all private keys between those values are equally likely. Is this not true? If people are going to attempt to brute force your key, don't you want the floor to start at 0? If the floor is higher, won't the brute force start higher too?
It's not that 4 isn't a valid seed (though insanely unlikely) it's that once I know your values weren't picked randomly because I guessed one of them I can easily figure out the others.
In this case one cert gets defeated and you go the odds of that number being that low are 2^-384. Essentially impossible if it were randomly selected. It's much more likely you didn't choose that at random so I start guessing a bunch of other low numbers for your other certificates and find them quickly.
Imagine you have a 3 digit PIN code. That’s 1,000 options to brute force. If you remove 000-009 as “too weak”, you still have 99% of the search space available. These numbers scale favorably as the private key size gets bigger.
In short, raising the floor doesn’t really hurt the search space meaningfully.
You are technically correct, the best kind of correct! Though I have a feeling we'll never see a 384-bit CSRNG generate 352 0-bits in a row in this Universe's lifetime.
That said, a sanity check on the value somewhere in the code would indeed have prevented this.
IIRC in RSA the key, a prime number, is selected between 2^(bits-1) and 2^bits, or iow it always starts with a 1₂.
Related: https://news.ycombinator.com/item?id=50166579
“eMudhra intermediate CA compromise” | 947 points | 19 hours ago | 408 comments
Of COURSE it's easy to crack, 4 isn't even prime!
In their defense, they were probably targeting the Mersenne prime 2^2 - 1 and just made a typo.
> In their defense, they were probably targeting the Mersenne prime 2^2 - 1 and just made a typo.
There are two hard problems in computer science: naming things, cache invalidation, and off-by-one errors.
Two, and only two?
You have almost understood the joke.
[dead]
ok, it's joke. but for those who don't get it: elliptic curves don't need primes
It’s really impressive no one caught their three previous private key values
I suspect they did but it wasn't publicised.
They should have gone with 57. Everyone knows it's prime.
Alex? You're alive after all?
It's been a while since this one popped up. It's surprising how many of those doodles are surviving the test if times, in the midst unexpected ways
The number 4 is very famous.
Hey, that's the same combination I have on my luggage!
Good job the Earth doesn't have an air shield protected by this byte or we'd be Skroob'd.
Well, so what now? It's not like I have these in my system CA store, they're just intermediaries cross-signed by sectigo.
Two more things to note about impact assessment: Flatpak uses the selected runtime's baked-in trust store (I think), so any apps may be susceptible even if you update the system package. Containers also don't inherit the host's trust store and use the one from the base image.
My izzat is down right now.
I wonder how someone stumbled upon this - maybe leaked by someone inside eMudhra?
Since there's nothing to indicate that the private key was all zeroes and one non-zero lsbyte, and presumably nobody would try to brute-force a private key.
I suspect that they read the article last Friday about Claude finding the origin of the ECDSA constants. That was only a few hours before Google received the vulnerability report.
The two cases are very similar. The only difference is that the weak randomness was harmless in the earlier story, and here it’s devastating.
Of the thousands of people bored with a swarm of AI agents, is it not plausible that somebody would just say "hey, let's do a vibe check to see whether any CAs are trivially crackable"?
This question seems to be answered in another comment now: https://news.ycombinator.com/item?id=50180284
> and presumably nobody would try to brute-force a private key.
Why not? There are examples of RNGs providing very poor entropy which might be a reason to check it (looking at you Sony).
Just seems weird for anyone to focus on trying to crack private keys starting with absurd 8-bit values that "nobody would use for a private key", let alone bother to do it at all, given the search space.
And here you learn why security is often just theater (not saying a proper elliptic key is theater)...
It took less than a second to factor this key. Why not attempt, say, to factor every CA, with a budget of less than a minute each?
The reward/risk ratio is skewed so heavily in favor of trying, the only reason not to is this idea that it is "impossible".
It's clearly not, hence, the theater. Your locks don't have to work, they often only have to look like they work...
I would not expect anyone to be on the lookout continuously, but like the researcher doing a "yolo" and trying. There are enough people out there that it is likely to happen at least once in a while. Or for instance someone looking for a firmware exploit in a downstream device.
I will say, there’s something almost reassuring when something goes boom that badly and the story is really silly. It’s like [horrific human-caused disaster] but it’s because of a Three Stooges episode.
The more cynical among us think that is part of it. Remember when Bush Jr. became a bumbling idiot for a few years?
1. So there was no certification of the certifiers?
2. This is stupidity that rises to the level of being malicious. There is no excuse, this was a fraud. Their business was literally to promise security and.. make a profit. Nothing was ever delivered.
> promise security and.. make a profit
Some would argue the inherent moral hazard makes these opposing outcomes that cannot be achieved simultaneously.
Like a web-of-trust model?
Someone used the Dilbert random number generator. https://www.random.org/analysis/
They clearly used the one from XKCD.
Could HN start feeding submissions through Pangram so us human readers don't have to suffer with reading the nauseating AI-prose?
Count yourself lucky - compared to Claude's prose, this is fairly readable.
It reads fine to me. If it is AI, I'd like to know which one so I can use it too.
It reads okay to me too, yet Pangram says it's mixed - the "What a private key actually is" section is AI, so is 'Why "it cannot be contained"' and some others.
I think it's a hybrid text. The section about how elliptic curve keys work is particularly aggravating. Unfortunately it's also the only section that gave me new information.
It has parts that are human written and parts that are generated. I couldn't tell at first, but now seeing it in Pangram, it's kind of clear.
It was all written by me and then I had final pass/tweak with Claude.
[dead]
That's some criminal negligence right there.
8 bits is crazytalk!
“4”
hey chatgbg what is a “private key”
As someone from Gothenburg, Sweden (often referred to as gbg) I shall answer:
A private key is a tiny number you don't need to share publicly because it is easy enough to guess.
Maybe they were counting on the fact that usually the right answer to “I’m thinking of a number between 1 and 10” is “7”.
Isn't 4 simply 2 bits?
Silly 2-bit outfit.
Maybe it's 3 bits.
I think the way that the bits are counted is based on the pool from which the number was randomly drawn. So it's possible they had 256 bits of entropy but got really unlucky.
I made test.key4.lol if you want to quickly check if your device/browser is vulnerable still.
It feels like they chose the private keys by hand.
Artisanal cryptography. Hand-picked private keys salted with organically sourced nonces.
Brooklyn-based. Guy in overalls and a waxed mustache hands out the bits.
Don't forget the leather apron, it's a messy job
Made with love
Sharing is caring after all.
My bet is on one of
- A sequence number being accidentally used as the private key or
- The private key being generated securely but then accidentally truncated to 8 bits (e.g. writing a single char into a zeroed buffer when you should've memcpy'd from a char*).
The sequence number idea is supported by the fact that two of the G3B private keys are consecutive numbers. I ran a quick Monte Carlo simulation and if I got it right, sampling 8 keys from [0,255] uniformly at random (as in the truncation scenario), you'd expect them to contain 2 consecutive numbers only about 18% of the time.
All the numbers also have digits going from left to right on a keyboard. I think the manual "random" input theory is plausible.
Oh wow, that is even more damning. Only 146 out of the possible 256 keys have this property. You only have about a 1% chance to get that 8 times in a row (A little more than 1% if sampling with replacement and a little less if sampling without. You'd expect them to at least not use the exact same key twice but at this point???).
Some error code being used? FIPS mode gone wrong?
There's a beginning of an explanation in the mozilla bugrepor thttps://bugzilla.mozilla.org/show_bug.cgi?id=2083449
Thanks. The relevant quote:
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
The fact that there are more than the "4" one, and all the low numbers are below 384 (384 being the bits for private key) makes me think whoever was creating these thought "Oh a random number between 1-384" let me choose... 4! (156, 230, 115, 56, 66, 237, 155)...
A long time ago one time pads used to be chosen by hand by women picking numbered lottery balls out of a spherical tumbling machine on a table.
This has a big upside: I can now MITM any Android app / IOT device without any suspicious set up on the device? (given it'll run on the unupdated OS and that cert revocation is not ON).
With the speed of vibe coding.. I'd expect the "Reverse Engineer Anything" folks to come out with a "Man in the Middle Anything" app by lunch
Question: what about the other trusted roots? Has anyone tested them?
I’ve tested most of them (from historical archives) and have automated testing any new CA certificates that show up in CT.
I’m calling the check “luggage code entropy” (1 byte).
It was a mistake to trust so many root CAs and misapply the same level of trust where clearly it is impossible to independently verify and most of the accountability is smoke and mirrors. We need to be asking “why” here, the stated reason needs to be much more than “because webtrust audit” and “we recorded the entire operational process for key material handling on YouTube with this open source implementation of key ceremony tooling and have underwritten the strength of the private key with a few $mm dollars, to be paid out to the rest of the roots for the shared reputational damage to the entire ecosystem should we fuck up (intentional or otherwise).
eMudhra explanation : https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11
Kind of hard to understand how a bug would have generated a second pair of key so bad, and how would human would have ended up using them. Never attribute to malice what can be attributed to stupidity, but sometimes you wonder.
Given the usefulness of this to anybody who knew about it between the ceremony and now - assume malice.
And an equal amount of stupidity in the whole ecosystem, apparently there are no "unit-tests" for trivial failure modes, like 8 bits of entropy, before adding a CA to the trusted list.
What next, googling the public key finds a 2 year old pastebin with the private key?
Even just a human glancing at the private key should do it.
Don't they even test script changes on a staging server? How could there be such a script error in production that was not caught in testing/staging?
This sounds very like a fundamental organization error, not an innocent whoopsie. It should lead to a permanent ban of the organization.
Well humans should NOT glance at private keys! That is the whole point of HSMs!
It seems there was some issue with the ceremony with some wrapping/unwrapping scripts.
They explicitly said they wanted to export the keys from the HSM; the keys were intended for USA/Europe (for sovereignty reasons) but were never actually used.
It seems the HSM itself worked, their ceremony around wrapping/unwrapping didn't.
There is something called "wrapping/unwrapping" - you encrypt the key, move it to the other HSM, and it unwraps it. So what moves around is encrypted. I don't know enough about HSMs to tell you if it's done using symetric or asymetric crypto, but no the key should not move around in the plaintext.
The write-up from eMudhra mentions something about wrapping/unwrapping ceremony failing.
Most HSMs used by CAs are pretty trash tbh.
Any sufficiently advanced malice is indistinguishable from stupidity.
The reverse is also true:
Any sufficiently advanced stupidity is indistinguishable from malice.
Never assume stupidity for what can adequately be explained by profitable malice.
Never assume unprofitable malice for what can adequately be explained by stupidity.
> recovered all of them from their public certificates on my laptop just now, and it took less than a second.
Why wasn't this second spend by the parties accepting those certificates as valid before they were accepted?
Shakti de. This should be firm-ending for them but I fear it won't be.
Man in the middle just for the love of the game
This is not a good write-up. Yesterday's discussion did contain better data, less self-Promotion and actually asked why it happened, instead of just saying "number small".
All the numbers were under 384. Surely they couldn't have ...
All the numbers were under 256. Surely they did.
I wouldn't be surprised if we learn in the near future, that this was deliberate.
I'm not so sure. If you wanted to be deliberate, you'd choose a plausible value that you could control.
No one is going to suspect `98761316832165974216498789746` is anything other than random. But `000000000000000123` looks suspicious.
But that would be obviously intentional. Much more plausible deniability with 0x4.
Why are many of them, suspicious?
The probability of getting a 0 is the same for every digit. However, in my opinion, the probability of getting a number that starts with two zeros is 1/10 * 1/10, and the probability of getting a number that starts with the same two digits is 10 * 1/10 * 1/10.
But the probability to receive one number or the other are the same.
By random chance you probably wouldn't get that many zeros. Of course, one could say it's just as random as any other string, but..
10 times the amount of digits is the possible number space.
smaller than 384 means 384 numbers (including 0) divided by the amount of possible numbers (see above) imho
Makes no sense.
If it was deliberate, one would use keys that only the person abusing it has. Not keys everyone can guess if they get a hint.
Deliberate deniability - if you leak a real key, it's pretty obvious you leaked a real key.
Here, you didn't leak anything, you just fucked up.
Why would it be?
Do I honestly need to enumerate what you can do if you have the keys for a CA?
Yeah, but if you are the one issuing the CA's private key, you already, by definition, have the keys to that CA. You don't need to make stupid keys in that scenario
But it doesn't make sense. Sure they'd find it extremely useful to have the key, but it doesn't need to be a weak key for it to be useful. If anything, it makes it less useful to them.
If I'm going to ask you or force you to share your password with me, there's no reason I'd ask you to set the password to 4. We might as well agree that the password would be D#!(s9fj0jalk!4af095SF42 , and it's much more likely our collaboration will continue working for years.
"Never attribute to malice that which is adequately explained by stupidity"
Sounds more like extremely sloppy procedures. Which depressingly I doubt are unusual among CAs.
As far as I know, to become a CA, with the requirements and ceremonies involved, this type of mistake must never happen. Assuming malice instead of ignorance is reasonable here. The entire premise of a CA, is trust.
There are a lot of CAs that are essentially grandfathered into root programs (and there’s no “one” root program, every OS and verifier gets to choose their own inclusion policies). They might need to jump through hoops to stay in any given program, but being there in the first place is not a strong indicator of quality.
This has to be a backdoor planted by hackers, mess with the key gen, say is all good. Must be known since issuance.
> and no TLS certificates chaining to these CAs ever showed up in CT
Why are browsers trusting CAs which issued zero certificates since their creation 1 year ago?
These are intermediate roots, signed by trusted root certs, so the trust from your OS or browser is implicit to the roots it trusts.
Yes, but there was no TLS cert issued that had these intermediate certs in its chain of trust at all anywhere. So why are they a CA if they don't issue any certs?
Their other intermediates have issued certs.
The CA concept itself is flawed, unless normal users were able to cherry pick the CA they do trust, dream. And even with that, they must keep in mind that many authorities and/or mafias have very probably rogue certificates for browser invisible man-in-the-middle.
It's an interesting idea, but how would a user make that decision on who to trust? As in, what 'things' could we look at to make that call?
Doesn't ACME+CT solve this threat?
Someone else has published usable .key files here: https://github.com/Fluepke/emudrahahahahaha
What the chances there’s another CA key out there with something like unix time as the key?
Another place to search for private key candidates would be the outputs of various common PRNGs when they have been seeded with a timestamp (for all timestamps in, say, the past 20 years).
These would look like nice random values (unlike the ones that were recently compromised), but they would be trivial to generate by anyone who knows the initial inputs (PRNG algorithm, seed, maybe starting byte position, etc.).
Literally the tutorial they used
Another very unfortunate even next to the php, fun times we are living in.
hey, at least it was easy to remember...
[dead]
Devices that are smart enough to talk to the internet but not smart enough to let the user install their own certs should not exist.
"let the user install their own certs should not exist"
are you sure that this is not opening to more attack vector ?????
Which is better - two attack vectors that can be mitigated by an expert, or one attack vector that can't be mitigated by anyone?
Users aren't supposed to be considered an attack vector. I don't care what the advertisers think.
The infected devices on the LAN are an attack vector, though.
Just imagine how much stuff you're going to be able to MiTM now!!
These devices are already very weak and many of them even do non-TLS updates with no signing check anyway.
If it has been so weak already, it can't be weaker.
Not always true. I have IoT devices where a valid certificate is useful because the original host doesn't exist anymore and the certificate isn't pinned.
Flashing the devices require direct UART/USB access which may involve soldering, if it even exists. This leak is gold for those because these devices will never get an updated CRL.
Some of them seem to do some cert pinning
> > Devices you can't update are the hard part. Routers, printers, TVs, appliances with frozen firmware will trust these certificates until expiry, which is July 2039...
The interesting question is, how many of these devices will have the compromised CA root certificate in place?
Oh and another thing is problematic, many a Java app ships with its own keystore.jks if it needs to trust some privately hosted endpoint, and that keystore.jks may or may not contain the compromised CA root certificate... updating the OS will not catch this kind of stuff.
[dead]