HN in RSCserver-reason-react
top.mdnew.mdbest.mdask.mdshow.mdjobs.md
← Back to stories

Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds

777 pointsby hkc 12 hours ago384 comments

Discussion

Loading discussion
  • alwa · 20 hours ago

    Bugzilla bug in question: https://.mozilla.org/showbug.cgi?id=2083449

    • LoganDark · 19 hours ago

      https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

  • pwfisher · 19 hours ago

    Test your browser and update your system's certificates: https://emudhra.we-have-a-problem.com/

  • unsupp0rted · 19 hours ago

    Never attribute to malice what can be explained by ignorance? In this case I wonder if this was an intentional wide open doorway to fraud

    • MiroslavPokorny · 18 hours ago

      This wasnt malice its was laziness. They literally were too lazy to type more than just a single '4'.

      • unsupp0rted · 16 hours ago

        Either that or they intentionally wanted to create a publicly-accessible backdoor, with plausible deniability of laziness

  • dlcarrier · 17 hours ago

    I thought CAs automatically filtered out really low keys. There's a great Suckerpinch video ( https://youtu.be/M1si1y5lvkk ) about trying to purposely make the least secure HTTPS page possible, and Tom initially tried to choose a laughably-low number as the key, but the CA automatically rejected it.

  • positive-spite · 12 hours ago

    That's some criminal negligence right there. 8 bits is crazytalk!

    • joshuat · 12 hours ago

      “4” hey chatgbg what is a “private key”

      • Hamuko · 11 hours ago

        Maybe they were counting on the fact that usually the right answer to “I’m thinking of a number between 1 and 10” is “7”.

      • Agentlien · 11 hours ago

        As someone from Gothenburg, Sweden (often referred to as gbg) I shall answer: A private key is a tiny number you don't need to share publicly because it is easy enough to guess.

    • bombcar · 10 hours ago

      Isn't 4 simply 2 bits? Silly 2-bit outfit. Maybe it's 3 bits.

      • patrickthebold · 10 hours ago

        I think the way that the bits are counted is based on the pool from which the number was randomly drawn. So it's possible they had 256 bits of entropy but got really unlucky.

        • thesz · 7 hours ago

          Their other keys are also in 0..255 range. This is definitely not "really unlucky." I guess they used (x `mod` 256) instead of (x `mod` (2^256)).

  • bombcar · 12 hours ago

    Of COURSE it's easy to crack, 4 isn't even prime!

    • joshuat · 12 hours ago

      It’s really impressive no one caught their three previous private key values

      • bcvvvy · 12 hours ago

        I suspect they did but it wasn't publicised.

    • DavidSJ · 11 hours ago

      In their defense, they were probably targeting the Mersenne prime 2^2 - 1 and just made a typo.

      • throw0101a · 11 hours ago

        > In their defense, they were probably targeting the Mersenne prime 2^2 - 1 and just made a typo. There are two hard problems in computer science: naming things, cache invalidation, and off-by-one errors.

        • zapkyeskrill · 9 hours ago

          Two, and only two?

          • amoss · 9 hours ago

            You have almost understood the joke.

          • floydnoel · 3 hours ago

            two is off-by-one from three btw

          • throw0101a · 49 minutes ago

            Well: There are actually only two hard problems in computer science: 0) Cache invalidation 1) Naming things 4) Asynchronous callbacks 2) Off-by-one errors 3) Scope creep 6) Bounds checking * https://news.ycombinator.com/item?id=38806257 Also: https://martinfowler.com/bliki/TwoHardThings.html

    • delibes · 11 hours ago

      https://xkcd.com/221/

      • lta · 10 hours ago

        It's been a while since this one popped up. It's surprising how many of those doodles are surviving the test if times, in the midst unexpected ways

    • walrus01 · 11 hours ago

      Hey, that's the same combination I have on my luggage!

      • wowczarek · 9 hours ago

        Good job the Earth doesn't have an air shield protected by this byte or we'd be Skroob'd.

    • wolfi1 · 10 hours ago

      ok, it's joke. but for those who don't get it: elliptic curves don't need primes

    • chompychop · 10 hours ago

      They should have gone with 57. Everyone knows it's prime.

      • kachnuv_ocasek · 10 hours ago

        Alex? You're alive after all?

      • bee_rider · 7 hours ago

        I wonder if anyone wants to over-analyze this joke to come up with the best candidates for the number? 57 falls too easily to the quick divisibility by 3 trick, IMO. (I realize over-analyzing jokes always makes them unfunny but we can pick the best number here and nobody has to know).

        • samatman · 6 hours ago

          Me! I'm in! Most-nearly-prime number under 100 is.... 91. 13 x 7. You've either memorized that or you're going to have fun deciding it's not prime. And if you did times tables up to 13, congratulations, you're special.

          • sejje · 2 hours ago

            7 is an easy check, not because i did them up to 13. because 10x7 = 70, i'm left with 21. everyone knows 21 is divisible by 7.

          • bee_rider · 1 hour ago

            I think 91 is a strong contender. Sejje has a way of working it out, but of course any small non-prime will have some trick to it.

    • Zeetah · 8 hours ago

      The number 4 is very famous. https://xkcd.com/221/

    • ohuf · 7 hours ago

      it doesn't have to be and it actually never is: this isn't RSA, it's an elliptic curve algorithm. if you'd have read the article, you'd have understood this

  • WalterGR · 12 hours ago

    Related: https://news.ycombinator.com/item?id=50166579 “eMudhra intermediate CA compromise” | 947 points | 19 hours ago | 408 comments

  • Qision · 12 hours ago

    It feels like they chose the private keys by hand.

    • PalmPilotProMax · 12 hours ago

      Artisanal cryptography. Hand-picked private keys salted with organically sourced nonces.

      • donandon · 12 hours ago

        Made with love

      • tclancy · 11 hours ago

        Brooklyn-based. Guy in overalls and a waxed mustache hands out the bits.

        • bigfishrunning · 8 hours ago

          Don't forget the leather apron, it's a messy job

      • account42 · 11 hours ago

        Sharing is caring after all.

    • muvlon · 11 hours ago

      My bet is on one of - A sequence number being accidentally used as the private key or - The private key being generated securely but then accidentally truncated to 8 bits (e.g. writing a single char into a zeroed buffer when you should've memcpy'd from a char*). The sequence number idea is supported by the fact that two of the G3B private keys are consecutive numbers. I ran a quick Monte Carlo simulation and if I got it right, sampling 8 keys from [0,255] uniformly at random (as in the truncation scenario), you'd expect them to contain 2 consecutive numbers only about 18% of the time.

      • account42 · 11 hours ago

        All the numbers also have digits going from left to right on a keyboard. I think the manual "random" input theory is plausible.

        • muvlon · 10 hours ago

          Oh wow, that is even more damning. Only 146 out of the possible 256 keys have this property. You only have about a 1% chance to get that 8 times in a row (A little more than 1% if sampling with replacement and a little less if sampling without. You'd expect them to at least not use the exact same key twice but at this point???).

      • Avamander · 10 hours ago

        Some error code being used? FIPS mode gone wrong?

    • walrus01 · 11 hours ago

      A long time ago one time pads used to be chosen by hand by women picking numbered lottery balls out of a spherical tumbling machine on a table.

    • Fabricio20 · 11 hours ago

      The fact that there are more than the "4" one, and all the low numbers are below 384 (384 being the bits for private key) makes me think whoever was creating these thought "Oh a random number between 1-384" let me choose... 4! (156, 230, 115, 56, 66, 237, 155)... https://xkcd.com/221/

    • gquere · 9 hours ago

      There's a beginning of an explanation in the mozilla bugrepor t https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

      • mminer237 · 8 hours ago

        Thanks. The relevant quote: > We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)

        • benmmurphy · 7 hours ago

          `a flaw in the code resulted in the second pair of keys being created onboard which were weaker` that sounds like the HSM is capable of generating keys in a range that is smaller than [1..group_order). such a feature makes no sense at all. or is this some kind of weird thing where people want to generate weak keys for 'testing' and HSM vendors have added such a strange feature. its actually satisfying to see someone get burned by this because such a feature should never exist and this is the reason why. if the HSM can generate keys in a smaller range (especially some arbitrary small range) then you are putting a lot of trust in the operators of the HSM. hopefully there are good audit logs and people actually check the audit logs every time keys are generated to make sure the keys being generated aren't from a special range.

  • naabozaurus · 12 hours ago

    hey, at least it was easy to remember...

    • Scott_Helme_ · 8 hours ago

      Can't lose your key if it's easy to remember!

  • jlmorton · 12 hours ago

    In retrospect, having 100 trusted certificate authorities might have been excessive.

    • tclancy · 11 hours ago

      In retrospect, not having a tiny competency check may have been an oversight.

      • oofbey · 2 hours ago

        Competence is difficult to check systematically. The problem is in the auditing process. The CA auditors check a much shorter list than an application that needs FIPS compliance. They just check that the CA is using a FIPS compliant key generator, and then basic things like following a logged, standard, multi person process. Anybody writing a system for a bank or the govt needs to have every line of code checked that it’s not doing something stupid with its crypto keys. That level of assurance would have caught this. But the process was defined too weakly.

        • tclancy · 1 hour ago

          Oh for sure. And at the time these were done, I'd guess cracking them was a lot more work. But still. 4.

          • knowinow · 59 minutes ago

            This key 4 was from a new ceremony, they must have hit the highest security key just recently and wrapped around.

        • InsidiousWombat · 18 minutes ago

          > They just check that the CA is using a FIPS compliant key generator, and then basic things like following a logged, standard, multi person process. FIPS-compliant key generator allowed key to be 4 ? No, they just didn't check thoroughly enough

    • BirAdam · 11 hours ago

      or, perhaps you need more so that one being stupid doesn't effect so much.

      • caaqil · 10 hours ago

        > perhaps you need more so that one being stupid doesn't effect so much. So you create more surface area for vulnerability? We might already be at or close to the sweet spot. Security is hard.

        • Gabrys1 · 9 hours ago

          I would assume that the client would need to validate the certs against more than 1 trust path (for the comment to make sense)

      • epanchin · 10 hours ago

        Each CA is like a padlock locked in series with other padlocks. The weakest padlock is the only one that matters.

        • akerl_ · 9 hours ago

          In a world with CT logs and CAA records, is that actually the case?

          • zbentley · 8 hours ago

            Yes, because as we are seeing with this incident, plenty of things don’t check CT logs (and thus can still be hacked if CT isn’t published for bad certs), and CAA records are fundamentally advisory.

            • akerl_ · 7 hours ago

              You said "yes" but then answered "no"? Like it seems your disagreement is that you're pointing out there are systems today that trust WebPKI CAs but don't validate CT/SCTs. I'd agree that that's an issue and I want those systems to join the world where CT exists. As I said, in a world with CAA records and CT logs, this isn't an issue. If your CA wants to be in root trust stores, you have to put your certs in CT logs. CAA records are "advisory" in the sense that we mandate that root-trusted CAs obey CAA records and that they write cert issuance to CT logs, so a trusted CA that violates CAA records has to tell on itself in a very detectable way.

              • Scott_Helme_ · 6 hours ago

                I'm just not sure that I can see a world where all devices that rely on TLS/PKI are capable of CT SCT validation. Look at the issues we have around stale root stores still happening when we have validity periods of literally decades. Some of these clients out there simply won't ever be able to keep their trusted log store updated. Refs: Part 1: https://scotthelme.co.uk/impending-doom-root-ca-expiring-leg... Part 2: https://scotthelme.co.uk/complexities-chain-building-ca-infr... Part 3: https://scotthelme.co.uk/cross-signing-alternate-trust-paths... Part 4: https://scotthelme.co.uk/building-certificate-chains/ There's also the Let's Encrypt root migration that was postponed due to legacy devices: https://scotthelme.co.uk/lets-encrypt-postpone-isrg-root-tra...

                • akerl_ · 6 hours ago

                  Similarly to vaccines, part of the magic is that you don't need "all", you just need "most". Right now CT support is rough for things outside of browsers; it's a pretty fair bet that if you pick up a command line tool or library, it is not checking SCTs, and that makes those workflows vulnerable. If we flipped that, and the average command line tool or library did validate by default, it would dramatically alter the risk calculus for attackers.

                  • Scott_Helme_ · 6 hours ago

                    For a scenario like this it doesn't help a given client what % of the ecosystem does SCT validation or not, all that matters is whether that client does it. I agree that more SCT validation across the ecosystem reduces the value of a mis-issued certificate to an attacker, but a lot of value still remains. If 90% of clients did SCT validation, an attacker can still succeed against the 10% that don't, there's no herd immunity.

              • lelandbatey · 6 hours ago

                You said: > In a world [where] CT logs and CAA records [exist] And we are in that world. CT logs and CAA records exist. They are around. What you are implying you said is: > In a world with CT logs and CAA records [and everyone/everything reads and validates them] We are not in that world, and we probably won't enter that world for the next 15 to 20 years because we are also in a world where folks expect networked devices to be able to communicate (mostly) securely without constantly communicating with a centralized set of clearinghouses arbitrating "what is secure". As we are discovering, our flawed systems may need broader rethinking, especially if we want secure communication (with all the epistemological ball of yarn that comes with it) without constant total centralization, which seems to be the approach-of-least-resistance lately.

    • otabdeveloper4 · 11 hours ago

      Correct, we only need zero.

      • sebzim4500 · 10 hours ago

        Interesting, what would be the alternative? I mean specifically for the use case of the browser warning the user if there is a MITM between them and e.g. their bank.

        • embedding-shape · 10 hours ago

          The first time you go to the bank, you do the "trust exchange" (maybe scanning a QR? People sure love that) that verifies future connections, similar to how you acquire digital certificates today when you deal with the government online and similar. Edit: > Every time you access your bank on different devices normalizes it so that visiting a typo domain would show the dialog and many would just click through. Show the dialog? What dialog? There is no dialogues here, you cannot login to the bank unless you've verified the connection. You do it with your phone, then you can propagate the trust from there to other devices if needed.

          • whywhywhywhy · 10 hours ago

            Every time you access your bank on different devices normalizes it so that visiting a typo domain would show the dialog and many would just click through.

          • bluehatbrit · 10 hours ago

            Not all banks have physical locations, not everyone has the ability to get to the ones which do. You can't walk back from the existing expectation and ease of use we have today. It's just not going to happen, people will opt for the current path because it's easy and seems good enough to them. Most normal people don't even know this whole thing is going on. They don't know the difference between http and https other than "it's secure". If we want to improve on this, it has to come with the current level of ease of use at a minimum.

            • monsieurbanana · 9 hours ago

              > If we want to improve on this, it has to come with the current level of ease of use at a minimum. OR it has to be a situation catastrophic enough. I don't think this one is, but it's hard to be optimistic for the future

            • __MatrixMan__ · 8 hours ago

              Presumably you had some reason to trust the bank in the first place. Maybe a recommendation from somebody you know? Get the key from them. The current level of ease is not being achieved securely. If it's not possible to do so, we should walk it back.

              • pixl97 · 4 hours ago

                > Maybe a recommendation from somebody you know? This works great for something that are obvious. "Hey, this plumber was good and cheap". The problem is this completely and totally breaks down where things are obscure. When things turn from "reals" to "feels" the amount of manipulation a bad actor can perform is great.

                • __MatrixMan__ · 34 minutes ago

                  Agreed, but what would you propose we do about it? The current pki just washes its hands of the situation and restricts itself to only validating server names. Which probably made sense back when the people running the servers were also the authors of the content those servers served, but now it's a false sense of security. I'm trying to do more than that. If we validate claims made by the content as anchored in the user's trust graph, we don't have to care which server it came from--it's trusted for different reasons. Are you saying we shouldn't try just because it's hard?

            • chrisjj · 7 hours ago

              > If we want to improve on this, it has to come with the current level of ease of use at a minimum. We need to recognise that is impossible. Making getting into a bank account easier for the owner does so for everyone else too. And also makes it easier for the bank to ditch the secure fallback - a building. And ask why vendor-indepedent trust shops aren't sprouting in every locale. Answer: same security problems. We are well and truly doomed to living in a scammers' paradise.

              • venzaspa · 7 hours ago

                But my bank has no physical buildings whatsoever? And also, how is my bank supposed to safely generate and distribute a key that we can exchange securely if an organisations who's primary purpose to generate these keys couldn't even manage to do so. I have plenty of other services I use too. Suddenly my phone company, doctors, supermarket has to create and distribute keys for me to go around collecting so that I can securely talk to them and we have to do this in a way that stops anyone from swapping those keys out before they get to me. I mean the current system isn't perfect, but it's a Utopia compared to that.

                • chrisjj · 5 hours ago

                  Yes. As I said, we're doomed.

          • u8080 · 8 hours ago

            Like...TOTP or FIDO2?

          • bee_rider · 7 hours ago

            That could be kind of interesting. It does seem to lean on the physical bank-front being trustworthy (no evil clerks). If we’re meeting physically to set up connections, we could even use something like a one-time-pad, and be information-theoretically secure. How many bits does your bank really need to send you securely anyway?

        • josephg · 10 hours ago

          Make dns secure and put certs / public keys in dns. Control over dns already gives you the right to get tls certs. It should just be connected directly. We don’t need CAs.

          • asksomeoneelse · 10 hours ago

            But how do you make DNS secure without CAs?

            • growse · 10 hours ago

              You just have one big one on the root. (Half joking).

              • asksomeoneelse · 9 hours ago

                Is that a reference to the Root Key of DNSSEC?

              • josephg · 8 hours ago

                Better to trust 1 organisation than trust 100.

          • shiandow · 10 hours ago

            Making DNS secure is not trivial though. In a way that's just moving the problem.

            • vesinisa · 10 hours ago

              Isn't this what DNSSEC solves?

              • muvlon · 10 hours ago

                DNSSEC deployment has been anything but trivial. This APNIC post from 2024 estimated that about 1% of DNS queries were being validated via DNSSEC: https://blog.apnic.net/2024/05/28/calling-time-on-dnssec/ . That's more than 25 years after the IETF first standardized DNSSEC.

                • giantrobot · 8 hours ago

                  2107: The year of IPv6, DNSSEC, and The Linux Desktop.

                • tptacek · 4 hours ago

                  I track DNSSEC adoption across the Tranco research list of top domains: https://dnssecmenot.fly.dev/

              • shiandow · 9 hours ago

                It should, though it achieves that goal by a chain of certificates not unlike CAs just tied to DNS. Like I said, it essentially just moves the problem. Which might still be better, if only because it limits the surface area.

              • SAI_Peregrinus · 2 hours ago

                No, DNSSEC tries to solve one part of the security of DNS (security of records between resolvers) and does nothing for the security between the user and their DNS server. And doesn't even do a good job at the bit it tries to solve.

            • josephg · 8 hours ago

              A secure DNS is valuable whether or not we get rid of CAs, since CAs rely on DNS queries to verify domain ownership. And if we have a secure DNS, then CAs are completely redundant. I'd much rather trust a handful of domain registrars than a handful of domain registrars, plus 100 random questionably competent organisations all over the world. Especially when any one of them can generate keys for every domain on the planet.

              • darkwater · 8 hours ago

                It seems a nice thing to have indeed, but domain registrars have already a history to cede control and bend to $GOVERNMENT. We have countries already trying to MiTM all their citizens by mandating CAs injection in every device under their jurisdiction, what would be the defense against this threat in a world where registrars hold the keys to TLS encryption for the web on the TLDs they own as root? I'm not that expert in this field so I really would like to learn if it's possible.

                • shiandow · 6 hours ago

                  Not that it isn't an interesting question but what's preventing them now? You can get a certificate for any domain you control. It is just really obvious if you abuse that power. Which shouldn't really change with DNSSEC I think.

                • GoblinSlayer · 4 hours ago

                  Government can compel servers in its jurisdiction to use a mandated CA. DNS would nicely compartmentalize it to the zone controlled by that government as opposed to the CA system where every random eMudhra CA has infinite reach.

              • tptacek · 4 hours ago

                "Secure" doesn't mean anything. A closer (though still imprecise) descriptor is the CIA triad --- confidentiality, integrity, and availability. A confidential DNS has real value. Without it, ISPs actively monitor customers and sell analytics about usage upstream; there are other problems, but virtually every US user faces that one. It's hard to imagine wanting to trade confidentiality off. A high-integrity DNS is a more dubious proposition. You'd obviously rather have integrity than not, ceteris paribus, but we have effective mitigating controls for DNS forgery, so you might imagine trading it off. Availability also sounds like a no-brainer, until you remember that almost every other component of the networking stack also has availability issues. You might not want to trade anything at all to get a more highly-available DNS, if you're going to be just as susceptible to DOS attacks because the current DDoS meta is application-layer anyways. There are a bunch of different approaches to securing the DNS, and they make different tradeoffs. DoH focuses on confidentiality, and has bank-shot benefits to integrity (because it breaks forgery attacks from on-path attackers). Confidentiality is its primary purpose and it trades basically everything else off. DNSSEC makes the opposite tradeoffs: it's exclusively for integrity, provides no confidentiality, and highly prioritizes availability (that's why it has elaborate authenticated denial systems). Depending on what you want to achieve, "secure" DNS can make sense and actually get deployed and used, as DoH is, or it can make almost no sense and see almost no uptake.

                • GoblinSlayer · 3 hours ago

                  Security provided by CAs is TLS Web Server Authentication (1.3.6.1.5.5.7.3.1). CAs too are exclusively for integrity, provide no confidentiality and don't prioritize availability.

                • InsidiousWombat · 7 minutes ago

                  DoH was mostly forced by few companies, it doesn't solve any real problems because the places that would like to spy on your DNS can just block few DoH providers that are used by default. Like, it's nice to have that random MITM can't see your DNS but if ISP wants it, they can just block it.

          • braiamp · 10 hours ago

            That still requires trust stores to be distributed to the clients and some clients don't need that baggage on the dns. Also, dns is by default insecure, dnssec is a configuration nightmare

            • otabdeveloper4 · 10 hours ago

              > That still requires trust stores to be distributed to the clients There's no such thing as a "trust store" in practice. To be "trusted" you just need to demonstrate you have control over a DNS TXT record, and we don't need a special "store" or intermediary for that. Certificate authorities are an artifact of a bygone time and an entirely different model of internet infrastructure.

              • woodruffw · 9 hours ago

                The trust store refers to the collection of parties that ultimately verify the signed claim of your control over a domain, which might have taken the form of a DNS TXT challenge (or any other number of challenges). You still need it, because there would otherwise be a basic circularity when determining whether to trust a connection. (Moving the PKI into DNS does not remove it, it just moves it around.)

                • josephg · 8 hours ago

                  > (Moving the PKI into DNS does not remove it, it just moves it around.) Better in DNS though. You could have certs mirror the structure of DNS. Install a single root signing key for the top level DNS registry on client devices. Then you can query that to find out the signing keys for .com, .net, .au, etc. And continue from there. One big problem with the CA system is that any CA - even the most incompetent - can sign certificates for any domain. If trust was moved to DNS, there would only be a single organisation that could forge certificates for any given domain. Using 7 as a private key? What a joke. The CAs aren't worthy of trust. We already have to trust the domain name registrars anyway, since CAs query them to figure out if you own a domain before signing a certificate. In total, removing CAs would not only simplify the system. But it would allow scoped signatures. And we'd end up needing to trust far fewer actors to have a secure internet.

                  • woodruffw · 8 hours ago

                    Why would I trust my domain registrar to maintain PKI? They don’t seem inherently better suited for it, and in various ways are more susceptible to meddling for e.g. ccTLDs. I think the idea that CAs are uniquely incompetent is probably incorrect.

                  • itintheory · 6 hours ago

                    > One big problem with the CA system is that any CA - even the most incompetent - can sign certificates for any domain. Is this not what CAA records were intended to solve? It seems like in practice they're only used by CAs to decide if they can sign a cert for a given domain, but it seems like a client could use it to determine if a server cert was from the correct CA.

                    • Scott_Helme_ · 5 hours ago

                      CAA records are only checked by the CA at the point of issuance, not by the client later. If they did, there are problems to consider: 1. A domain would have to keep the CAA records for a CA in place to keep using the certificates from that CA, but that means continuing to authorise new issuance too. Right now, I can remove the `letsencrypt.org` value to stop them from issuing new certificates for me, but I can keep using any certificates I have until they expire. 2. The client would need to have support for this new DNS RR type. 3. The client would need a secure way to query DNS, which is not widespread.

                    • parasyte · 5 hours ago

                      CAA records were never intended to be used by clients (Relying Parties), the RFC specifically prohibits them from using the CAA record to determine if the issuer was allowed to have issued the cert. I infer from the wording around there and elsewhere in the RFC that this is mostly because someone could switch issuers but still serve valid certs from the prior issuer, and there's no way an RP would have a historical record of which CAA values were in effect when to be able to determine if the issuance was valid or not.

          • three14 · 9 hours ago

            tptacek argues that DNSSEC is too flawed to actually do this: https://sockpuppet.org/blog/2015/01/15/against-dnssec/

            • ranger_danger · 9 hours ago

              I have seen their comments across this site for a very long time. Unfortunately I do not respect or put any stock in what they say, because of how unbelievably rude and dogmatic they are, and always have been. "As a rule, strong feelings about issues do not emerge from deep understanding."

              • icedchai · 8 hours ago

                Fortunately, you can investigate this stuff for yourself and develop your own opinion. IMHO, DNSSEC doesn't actually "secure" much of anything. It can be disabled trivially by anyone operating a resolver. As I mentioned elsewhere, none of the major sites (FAANG) use it. What does that tell you?

                • ranger_danger · 7 hours ago

                  Yes I'm not saying DNSSEC is or isn't good. I think a multi-layered approach would be best for redundancy.

                  • tptacek · 4 hours ago

                    Wait. Hold on. It is either good or it isn't. If it's good, we should be using it (maybe in a layer cake, maybe a la carte). If it's bad, it should not be used anywhere.

                • GoblinSlayer · 4 hours ago

                  Then propose a censorship resistant protocol.

            • josephg · 8 hours ago

              He's right. DNSSEC is a mess, and we should fix that. We should fix that whether or not we delete CAs because CAs rely on DNS to verify that you own your domain. If DNS isn't secure, neither are CAs. In the long run, fixing bugs is almost always better than working around them. DNSSEC is a bug. So are CAs. We can and should fix both.

              • chimeracoder · 7 hours ago

                His argument is that DNSSEC is fundamentally broken, and you can't fix it via incremental improvements. He is not arguing that DNS security cannot or should not be improved; in fact, he explicitly states the exact opposite.

                • tptacek · 6 hours ago

                  I don't have an objection to doing a different DNS security system, and in fact I like DoH a lot. You can do DoH all the way back from authority servers if you like, and that addresses domain validation without boiling the ocean with 1990s cryptography the way DNSSEC tries to do.

            • GoblinSlayer · 7 hours ago

              ECDSA was added to DNSSEC 3 years before that blog post in april 2012 (rfc6605).

              • tptacek · 6 hours ago

                And then for the next 10 years APNIC wrote posts about how actually using ECDSA broke resolvers. It's going to settle into general usability at just about the time cryptographers deprecate ECC entirely!

                • GoblinSlayer · 3 hours ago

                  They can be deprecated for confidentiality purpose, but DNSSEC doesn't use confidentiality.

                  • tptacek · 3 hours ago

                    You're trying to argue that DNSSEC isn't exposed to HNDL attacks, so can use signature algorithms that CRQCs break. But that's neither here nor there: the approaching cryptographic research consensus is going to deprecate all ECC (and all RSA); yes, signatures aren't the priority for immediate removal, but nobody is recommending ECC signatures. The point is all this is happening just as ECC in DNSSEC may finally be stabilizing.

          • jayd16 · 5 hours ago

            So like, the cable company now controls the certs for most consumers? shudders

            • tptacek · 4 hours ago

              Yes. This is basically the DANE pitch. Subtextually, supporters argue that two big shifts will immediately happen: (1) We're also rearchitect the way all DNS is done so that every lookup from every phone and laptop is fully recursive, so that everyone validates DNSSEC chains instead of relying on the AD bit that cable company DPI devices switch off. (2) People will lose all attachment to their current domain names and simply migrate to TLDs that have policies they agree with, rather than .COM and .IO, which are under the demonstrated de facto control of the NATO IC.

          • 1vuio0pswjnm7 · 1 hour ago

            Perhaps browser fans will check the source code to verify that their browsers or operating systems using DoH ("Private DNS" or whatever they call it) check the certificates presented by DoH providers against CT logs The dependency of "accetpable", i.e., not self-signed, TLS certificates on DNS, i.e., a domain name in a TLD created by "ICANN", is funny when one considers that the traffic to authoritative nameservers is still unencrypted. It stands to reason that DoH and other "accepted" forms of "DNS security" are only to stop ISPs from stealing Silicon Valley's surveillance advertising racket. Neither is "DNSSEC" providing the user with any real "security"; it's protecting a monopoly. The "security" is against anyone who might try to use domains names not blessed by "ICANN" Indeed there's no need for CAs just as there is no need for ICANN

        • lukewarm707 · 10 hours ago

          For me the complexity of anything connected to public internet is now over what I can secure. I genuinely can't say I secured a PC, when it needs to phone home for a configuration at this point. We are most likely over the line for needing to airgap a personal PC.

        • Ohentis · 9 hours ago

          I quite like how i2p does it. Public keys are part of addresses, human readable names are user defined and shared but static.

        • __MatrixMan__ · 8 hours ago

          Stop validating names, start signing content. Scan a QR code when you open the bank account, add it to the keys you trust as authorities on financial things. When your browser runs accross data tagged "financial" it checks the trusted keys in that domain and renders it only if it was signed by a key that is trusted in that domain (the domains prevent your bank from abusing that trust by causing you to see political ads or some such). If you ever see something that you suspect is malicious, you untrust whichever key signed it and take it up with whoever asked you to trust their key. If you verify it's malicious, gossip the bad keys to your friends so they can distrust also.

          • diarrhea · 8 hours ago

            That solves integrity but not confidentiality. It does not protect against MITM.

            • __MatrixMan__ · 5 hours ago

              It does protect against MITM. A man in the middle can pass the content along or they can refuse to do so but they can't forge a signature. If confidentiality is required, well you already have a public key that you're using to verify content, then the content can say: "reach us at 196.7.3.45" and then you encrypt communications to that address with the same public key you use for validating content. Or maybe they'll specify, in the content, that you should use a different key for that, but either way the chain of trust is preserved. This would of course all be protocol level stuff, handled by the browser, not a human. They just have to maintain a list of keys they trust.

              • pixl97 · 4 hours ago

                >It does protect against MITM. A man in the middle can pass the content along or they can refuse to do so but they can't forge a signature. Um, I don't think you understand this problem near as good as you think you do? >A man in the middle can pass the content along or they can refuse to do so but they can't forge a signature. So where do you get this signature exactly so they don't MITM you the first time? > well you already have a public key that you're using to verify content, Who what where when why? Where did I get this public key? Who provided it? Is this my public key or their public key? How do I trust it? >They just have to maintain a list of keys they trust. So you've solved exactly nothing.

                • __MatrixMan__ · 2 hours ago

                  > So where do you get this signature exactly so they don't MITM you the first time? That's always going to depend on why you're trusting them in the first place. If somebody handed you a business card the key should be on the card. If it was a recommendation from a friend you should use whatever key they use. If you're expecting to just type random IP addresses into your browser until you hit something and then be told that you should trust what you see... Well that's not in the cards because you shouldn't trust what you see in that scenario. We already don't type in URL's, we send links to each other. It really wouldn't be so difficult to change the UX so that when we delegate trust in that way it's public keys that are being trafficked instead. > So you've solved exactly nothing. Removing untrustworthy third parties and the need to revoke trust in them on the user's behalf is the solution. Users don't know what a CA is or how they should evaluate its trustworthiness. We need a system where they're instead evaluating the trustworthiness of things they understand, like other people. When they get burned they should have the ability to immediately revoke trust in whoever burned them. This thing we're doing where we expect them to wait around while some nerd pushes the updated list to them and to blindly trust that that nerd isn't curating that list with ulterior motives is totally bonkers.

          • chrisjj · 8 hours ago

            > When your browser runs accross data tagged "financial" it checks the trusted keys 'Cos no scammer would ever be so sneaky as to leave off the tag... /s > it up with whoever asked you to trust their key 'Cos no scammer would ever pretend to be that person... /s

            • __MatrixMan__ · 4 hours ago

              Well if you find unsigned content, you should probably just not show it to the user at all. As for tagging... We could use something like color to indicate domain. So if your well trusted plumber is giving financial advice maybe it shows up, but not in the "financial" color? Idk it seems like a solvable UX problem.

              • chrisjj · 3 hours ago

                Anything relying on user recognising some tag colour seems like an unsolvable UX problem to me.

          • lokar · 6 hours ago

            So, have each domain run its own root CA (scoped to the domain), and have users trust them one at a time when they have cause to interact with them?

            • __MatrixMan__ · 5 hours ago

              Yes to smaller scopes, with keys passed of during interactions that organically arise, preferably face to face, but maybe not always. No to making domain names be the thing that's being trusted. We should push this into the application layer, so rather than scoping the keys to domain names let's scope them to something relevant to whatever the user authentically trusts. Something to do with their intentions for using this app. Because most users don't even know what a CA is, but they understand that most of the time they'll need to trust somebody. So let the app decide what that's going to mean because those dynamics are always going to depend on the situation. So the user (or OS) would pass an object at runtime to indicate which keys they want the process to trust for this session. Typically it'll be another human or a company, not a server name, but I suppose it could also be a device. Something that can guard a key pair and be identified by its ability to produce valid signatures. That way you're just not relying on globally unique human friendly names at all. Doing so puts you in a cursed corner of zooko's triangle. Users can assign a pet name for whoever they're trusting at the time they're configuring that trust since that's the moment they're best able to scrutinize the trustworthiness of this individual or company or device. Not later on when they're installing software updates and have no context for the decisions they're making. This way your application can work over alternative transport layers: carrier pigeons, lobbing an SD card over the border wall, whatever, and doesn't break just because the other party has a different name on the new transport. Relevant for when the internet goes down and you still want it to work with other hosts on the LAN or the local mesh or whatever.

            • xp84 · 1 hour ago

              In the modern day that's arguably fine, most users only interact with a few actual companies online (though many of them haphazardly use dozens of domains and dozens of unrelated certificates). The only 'real' problem though with this approach is it requires a trusted path for the bank to get the "JP Morgan Chase root CA cert" into my hands (like a QR code they hand me physically). Serving it up over http: or over an unvalidated SSL context at first use seems to defeat the purpose since I could be being MitMd the first time I hit chase.com. In practice most people would be angry that they couldn't use the bank's website if their new phone or PC didn't have the certificate yet, and if places start advertising their cert QRs everywhere, it will normalize the idea that you add a CA certs all the time, leading to CA cert QRs being on billboards and bus stops and eventually fraudulent ones would be snuck in that way.

          • tptacek · 3 hours ago

            You are describing key pinning. You have invented QR-code-based key pinning.

            • __MatrixMan__ · 1 hour ago

              My intent is to provide users with the ability to revoke trust if in a source of information if they realize it is malicious, and to gossip that revocation among their peers. RFC 7469 appears to be more geared towards service providers attempting to protect their users. Laudable, but we're still assuming we know something about who the user deems trustworthy, which I think is presumptuous. It's one thing to suss out cases where somebody is lying about their name, that's pretty cut and dry. But ever since web2.0 there's no longer much of an association between the hostname and the trustworthiness of the content. If we instead provided a means of validating the content's provenance it wouldn't much matter which host was involved in its delivery.

        • rocqua · 7 hours ago

          In practice we have Certificate Transparency Logs that let us check whether a CA has issued a certificate for a domain you control without you asking them to. This really reduces how much we need to trust the CAs, because we can detect any untrustworthy actions after the fact. Now detecting after it happened is too late. But it does mean that mistakes or malicious actions are much more expensive to a CA. In practice this has really reduced the risk, especially to those whom aren't interesting enough to burn a valuable capacity on. Which is the vast majority of people. Though perhaps if you count TLS traffic weighted by importance that count is different.

          • zaptheimpaler · 3 hours ago

            I think someone explained that only browsers and maybe some applications check the CT logs in the first place. Most applications don't, so someone can impersonate any URL an application connects to and the attacker doesn't have to publish to a CT log.

          • krackers · 1 hour ago

            How would certificate transparency logs have helped here though? If the private key leaks and anyone can mint a valid certificate that gets accepted, those logs aren't going to reveal it.

        • InsidiousWombat · 12 minutes ago

          Zero would be hard but we could have signed CA cert put into DNS record and from that the org itself can create the certs for that domain. Then at the very least one of the CAs leaking isn't automatic compromise, as leaking the key still doesn't make you insecure, that would also require intercepting and changing DNS records

    • m3kw9 · 8 hours ago

      How are they trusted if they can just generate any key they like in their back room?

      • bigfishrunning · 8 hours ago

        That's the thing about trust -- in order to get into the root key store for windows, chrome, linux, etc they were trusted not to do that. they violated that trust.

      • swores · 8 hours ago

        "Trusted" means how people treated them, it doesn't mean the extent to which they deserved trust.

        • chrisjj · 8 hours ago

          This. Good: trustworthy Often bad: trusted

      • RALaBarge · 6 hours ago

        There are chains of trust which are established for SSL certs, centered around the idea that if we can believe Mozilla et al. that the server responding is who they claim they are. Up until now, that is good enough to establish the trust needed to accept the 2 keys in the connection are connected, and then a secure connection could be established to the host server. Mozilla et al. could claim trust for Microsoft and then MS would be an intermediary for Mozilla. There is a long history of sketchy intermediaries like GoDaddy and others that root certificate providers like Mozilla would block at times due to abuse. This leads to users, usually ones that are not that technically savvy, seeing 'insecure' warnings on a page that actually offers HTTPS. ..but since the root servers no longer accept them as trusted, everything that GoDaddy supported, and every intermediary (child) cert which they approved would no longer be trusted and you would see a non-secure warning.

        • BubbleRings · 49 minutes ago

          Which leads to users that are not that technically savvy learning to always click "Load it anyway!" and not think about it, even in financial management firms. Ask me how I know...

    • Scott_Helme_ · 8 hours ago

      That really depends how you define "trusted certificate authorities". If you're talking about root certs then it's almost 350 currently trusted by at least one major operator. If you include intermediates, you're talking more than 3,600!

    • londons_explore · 5 hours ago

      Certificate transparency requirements effectively take the power away from certificate authorities. They instead place the burden on individual webmasters again to actually verify that nobody has been falsely issuing for their domains. There are also some new vulnerabilities about CT logs who might be malicious.

      • Scott_Helme_ · 5 hours ago

        I agree this is a burden placed on domain owners, but who else can know if a certificate for my domain 'scotthelme.co.uk' was supposed to be issued other than me, the owner of 'scotthelme.co.uk'? Do you have any info on the malicious CT logs? I'd be interested to read something on that!

        • jasomill · 1 hour ago

          Possibly CT logs revealing information about domain name and IP address spaces, or other information through implementation details, that might not otherwise be public.

        • londons_explore · 1 hour ago

          CT logs stapling proof, but then never actually including that cert in the log.

  • maeln · 12 hours ago

    eMudhra explanation : https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11 Kind of hard to understand how a bug would have generated a second pair of key so bad, and how would human would have ended up using them. Never attribute to malice what can be attributed to stupidity, but sometimes you wonder.

    • sobkas · 11 hours ago

      Any sufficiently advanced malice is indistinguishable from stupidity.

      • SAI_Peregrinus · 9 hours ago

        Never assume stupidity for what can adequately be explained by profitable malice. Never assume unprofitable malice for what can adequately be explained by stupidity.

      • RetroTechie · 9 hours ago

        The reverse is also true: Any sufficiently advanced stupidity is indistinguishable from malice.

    • dist-epoch · 11 hours ago

      And an equal amount of stupidity in the whole ecosystem, apparently there are no "unit-tests" for trivial failure modes, like 8 bits of entropy, before adding a CA to the trusted list. What next, googling the public key finds a 2 year old pastebin with the private key?

      • silvestrov · 11 hours ago

        Even just a human glancing at the private key should do it. Don't they even test script changes on a staging server? How could there be such a script error in production that was not caught in testing/staging? This sounds very like a fundamental organization error, not an innocent whoopsie. It should lead to a permanent ban of the organization.

        • karel-3d · 10 hours ago

          Well humans should NOT glance at private keys! That is the whole point of HSMs!

          • consp · 10 hours ago

            I'm still flabergasted the HSM allowed this key to be generated. Then again, I've worked with HSMs, there are always many mistakes in the firmware, usually less severe though.

            • karel-3d · 10 hours ago

              It seems there was some issue with the ceremony with some wrapping/unwrapping scripts. They explicitly said they wanted to export the keys from the HSM; the keys were intended for USA/Europe (for sovereignty reasons) but were never actually used. It seems the HSM itself worked, their ceremony around wrapping/unwrapping didn't.

              • consp · 9 hours ago

                Makes more sense. I'm sure the Utimaco's used (haven't worked with one) also allow all slots to contain the same name or keys with the same name and the device always selecting the first one if you only provide the name and not the slot ID or token identifier. Don't ask how I found out the hard way.

                • karel-3d · 9 hours ago

                  Yes there was something about key names in the write-up (which I cannot find now) oh here https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11

          • silvestrov · 9 hours ago

            But eMudhra wanted to (was required to) export the keys. So using an HSM in the first place is a security theater. Or that this HSM isn't really an HSM since it allows for export of the private key. The trusted enclave in a MacBook is more secure than this HSM.

            • xvector · 8 hours ago

              Most HSMs used by CAs are pretty trash tbh.

            • karel-3d · 8 hours ago

              There is something called "wrapping/unwrapping" - you encrypt the key, move it to the other HSM, and it unwraps it. So what moves around is encrypted. I don't know enough about HSMs to tell you if it's done using symetric or asymetric crypto, but no the key should not move around in the plaintext. The write-up from eMudhra mentions something about wrapping/unwrapping ceremony failing.

    • amoss · 9 hours ago

      Given the usefulness of this to anybody who knew about it between the ceremony and now - assume malice.

      • palata · 36 minutes ago

        If I was to leak a private key for malicious use, I would leak a good key that only I know.

  • siwatanejo · 12 hours ago

    https://xkcd.com/221/

    • Qision · 11 hours ago

      Literally the tutorial they used

    • Scott_Helme_ · 8 hours ago

      I think this might be proprietary source code, did you get permission to share this?..

  • TazeTSchnitzel · 11 hours ago

    Could HN start feeding submissions through Pangram so us human readers don't have to suffer with reading the nauseating AI-prose?

    • jadamson · 11 hours ago

      It reads fine to me. If it is AI, I'd like to know which one so I can use it too.

      • TazeTSchnitzel · 11 hours ago

        I think it's a hybrid text. The section about how elliptic curve keys work is particularly aggravating. Unfortunately it's also the only section that gave me new information.

      • stratos123 · 11 hours ago

        It reads okay to me too, yet Pangram says it's mixed - the "What a private key actually is" section is AI, so is 'Why "it cannot be contained"' and some others.

      • karel-3d · 10 hours ago

        It has parts that are human written and parts that are generated. I couldn't tell at first, but now seeing it in Pangram, it's kind of clear.

        • Scott_Helme_ · 8 hours ago

          It was all written by me and then I had final pass/tweak with Claude.

          • queenkjuul · 4 hours ago

            So parts were generated... Unless you're saying Claude just went "lgtm, ship it"

            • Scott_Helme_ · 4 hours ago

              Fair enough, call it a mix if you like. I'm happy to stand behind every claim in my post, and if something's wrong, tell me and I'll fix it.

    • swiftcoder · 11 hours ago

      Count yourself lucky - compared to Claude's prose, this is fairly readable.

  • gagan2020 · 11 hours ago

    "Importantly, none of the affected intermediate CAs were deployed in production, with the exception of one. Furthermore, none of these intermediate CAs, including the one deployed in production, have issued any end-entity certificates. This has been confirmed through our internal analysis and corroborated by publicly available Certificate Transparency records on crt.sh." This statement is more discerning than anything else. It raises question about why this certificates exists in first place if not deployed since 2025. Is it deliberate attempt to do compromising of system? In recent times, Indian Government is becoming quite rogue and don't follow the said procedures. So, it might be related to that as well.

  • hypfer · 11 hours ago

    This is not a good write-up. Yesterday's discussion did contain better data, less self-Promotion and actually asked why it happened, instead of just saying "number small".

  • tclancy · 11 hours ago

    I will say, there’s something almost reassuring when something goes boom that badly and the story is really silly. It’s like [horrific human-caused disaster] but it’s because of a Three Stooges episode.

    • skinfaxi · 9 hours ago

      The more cynical among us think that is part of it. Remember when Bush Jr. became a bumbling idiot for a few years?

  • sscaryterry · 11 hours ago

    I wouldn't be surprised if we learn in the near future, that this was deliberate.

    • yoavm · 11 hours ago

      Why would it be?

      • sscaryterry · 11 hours ago

        Do I honestly need to enumerate what you can do if you have the keys for a CA?

        • swiftcoder · 11 hours ago

          Yeah, but if you are the one issuing the CA's private key, you already , by definition, have the keys to that CA. You don't need to make stupid keys in that scenario

          • sscaryterry · 10 hours ago

            In case it wasn't obvious. The NSA and many other organisation would find this extremely useful.

            • yoavm · 10 hours ago

              But it doesn't make sense. Sure they'd find it extremely useful to have the key, but it doesn't need to be a weak key for it to be useful. If anything, it makes it less useful to them. If I'm going to ask you or force you to share your password with me, there's no reason I'd ask you to set the password to 4. We might as well agree that the password would be D#!(s9fj0jalk!4af095SF42 , and it's much more likely our collaboration will continue working for years. "Never attribute to malice that which is adequately explained by stupidity"

              • skinfaxi · 9 hours ago

                A weak key means use is unattributable.

                • NetMageSCW · 6 hours ago

                  A secret strong key is also unattributable.

    • edent · 11 hours ago

      I'm not so sure. If you wanted to be deliberate, you'd choose a plausible value that you could control. No one is going to suspect `98761316832165974216498789746` is anything other than random. But `000000000000000123` looks suspicious.

      • sscaryterry · 11 hours ago

        Why are many of them, suspicious?