Your wallet stores the secret (or private) key and your public address to interact with the money on blockchain securely. There are two kinds of wallets for storing keys:
Cold Wallet (wallet which is not connected to internet):
Cold wallet is an offline wallet provided for storing bitcoins. With cold storage, the digital wallet is stored on a platform that is not connected to the internet, thereby, protecting the wallet from unauthorized access, cyber hacks, and other vulnerabilities that a system connected to the internet is susceptible to.
The most basic form of cold storage is paper wallet. A paper wallet is simply a document that has the public and private keys written on it. The document is printed from the bitcoin paper wallet tool online with an offline printer. The paper wallet or document usually has a QR code embedded on it so that it can easily be scanned and signed to make a transaction. The drawback to this medium is that if the paper is lost or destroyed, the user will never be able to access his address where his funds are.
Another form of cold storage is a hardware wallet which uses an offline device or smartcard to generate private keys offline. The device looks and functions like a USB, and a computer and chrome-based app are required to store the private keys offline. Like a paper wallet, it is essential to store this USB device and smartcard in a safe place, as any damage or loss could terminate access to the user’s bitcoins. Two popular hardware wallets are TREZOR and KeepKey. These are expensive to buy.
Another form of cold storage is offline software wallets on desktop or mobile. They are mostly offline and pretty secure as long as your computer is not hacked. Electrum, Exodus and Armory are often quoted as the best offline software wallets in this category.
Hot Wallet(wallet that is connected to internet):
Private keys stored on a hot wallet are connected to the internet are vulnerable to network-based theft. These wallets are known as hot wallets. Here all the functions required to complete a transaction are made from online device. The wallet generates and stores private keys; digitally signs transactions using private keys; and broadcasts the signed transaction to the network.
It is considered "unsafe" because hackers have easier access to this than a cold wallet. So a minimal amount is stored on a hot wallet. The exchanges are the type of hot wallets. Examples: Coinbase and Blockchain.info
Generally as a rule of thumb you should only leave as much money on your hot wallet as you would with a leather wallet that you’d keep in your pocket. Think of it this way, if you were held at gunpoint while holding a leather wallet, then you’d only lose that money in your pocket, and not your entire bank account. If you keep all your money in Coinbase it’s as if you are walking around town with all your money in your pocket.