If I use a hardware wallet with multiple cryptocurrencies, then I guess it uses multiple addresses to store them, right? Now I'd like to back up this wallet on a piece of paper in case it's destroyed. Can the 24-word sentence created by the hardware wallet once initialized be used to recreate all these addresses, across all currencies on the device?
BIP32 and BIP44 define a standard for Hierarchical Deterministic Wallets with Multi-Account Hierarchy. While these are originally Bitcoin standards, they can be adopted by wallets for other cryptocurrencies and are especially useful for hardware wallets that are meant to be able to work with many different cryptocurrencies at once.
To answer your question, yes, these standards are indeed able to deterministically generate all your addresses for multiple cryptocurrencies from a single master seed (usually with the use of a BIP39 mnemonic phrase).