By cold wallet, I mean any type of encrypted USB etc. If this isn't possible how would I buy crypto and then be able to store it on a USB in the first place? This may be a stupid question but I'm kind of new to this stuff.
The coins are not stored on the USB drive, they keys that can spend the coins are.
An 'encrypted USB' would likely not constitute a 'cold storage' device, that term is usually used to describe keys which have never existed on an online device. If you use your (online) computer to generate keys, and then store them on a USB drive, that would not constitute 'cold storage'. Generating and storing keys offline is a best practise, because it reduces the risk of an attacker stealing your keys over-the-wire.
So a lot of your security comes from how the keys are generated, as well as how they're stored. 'Hardware wallets' are devices that are built to provide a mix of security and convenience that many users find appealing, if you are new to all of this, they might be a good choice. For the more technical user seeking extreme security, there are more advanced setups possible (sanitized general-purpose hardware, multi-sig using multiple devices, etc).
Once you've purchased coins, you should be able to initiate a withdrawal from the exchange service, by copy/pasting your 'receiving address' from your wallet software, into the 'send/withdraw/etc' field on the exchange service's website.
Note that some services do not allow you to withdraw your bitcoin (eg, paypal, robinhood), so these are obviously terrible places to purchase bitcoin, that should be avoided at all costs.