There is currently no way I could find that someone could extract a private key out of a Ledger wallet. So if someone steals your Ledger today, they should not be able to spend your coins.
However, since the private keys are physically stored in the device, someone successfully extracting them isn't a question of "if" but "when". In case of a firmware vulnerability it may be publicly disclosed only after it is patched with a firmware update, but whoever stole your Ledger simply won't install the update and will now have access to your funds.
In other words, if you lose your Ledger wallet (or any other hardware wallet) or it gets stolen, it's best to sweep your funds into a wallet with a new seed, sooner rather than later.