Keep in mind that the ledger itself, if compromised could leak the pk in other ways. These methods I outlined will only confirm to you that a given address belongs to the set of private keys you hold, not that you're the only person holding these pks
like you said: one way you can cheaply verify that is to simply take the address and send a little bit of money to it. That does not answer if your key is compromised but it does tell you if the address belongs to your seed or not.
Another way would be to manually derivate the address from the private key using the BIP39 tool here until you reach the address that ledger gave you.
This second one doesn't require you to send any money but it is more risky as you need to trust the code in the page I sent you and you're literally inserting your pk in a form in the web, which is never a good idea.
The third thing you could do is generate Private Keys with a method you trust (i.e. using Samourai wallet or something) and then use that mnemonic in the ledger as a seed. This way you know that only you have that private key.