When you start using a cryptocurrency hardware wallet, you get to write down a seed phrase in some physical medium (paper, metal, etc.) so you can regain access to your funds if you lose the hardware wallet.
If anyone who gains access to this seed phrase can regenerate your private keys in another wallet and get access to your funds, why use a hardware wallet in the first place? Why not just keep the seed phrase well hidden, remove your data from any wallet app and use the seed phrase to regenerate your keys anytime you need to transfer funds? Is it only a matter of convenience and security when actually transferring the funds?