Ledger uses a secure element chip to store the private key.
In simplified terms this means that the private key is stored on a special chip that is almost like a small computer with a Operating System with a very strict and well defined interface. This interface is designed to NEVER allow an extraction of the private key except during the creation of the key when the seed is shown on the display. The only operation that uses the private key that is supported is to send input (read transactions in terms of bitcoin) that the chip signs with the private key, and then returns the signature. Therefore the only output you could get from the chip is the signed transaction, and never the private key, no matter where you put the chip.
Secure element chips are not a new invention by Ledger, but actually a technology used in many other use cases (other examples are passports and credit cards). Ledger uses a Javacard with version 3+, which supports creation of ECC keys that have size of bitcoin keys.
It should however be noted, that even though the chips are designed to never let anyone extract the key from the chip, even with physical access to it, a well motivated and funded actor (like NSA) could theoretically extract the private key.