The blockchain is very large and takes a lot of time to download to a client. Apparently, the only thing we need from the blockchain is the balance of each "user" (- public key). So, instead of keeping the entire blockchain, we could keep a smaller ledger that only contains the balances. Why isn't this done? What are the advantages of keeping the entire sequence of transactions?
Keeping the entire transaction history allows us to independently verify that everyone's balances (note that Bitcoin does not actually use accounts or balances) are correct. Without having the full transaction history, we cannot verify whether someone actually had the money; the only way to do so is to follow the trail of transactions all the way back to the creation of a coin and check that all of those transactions are valid.
Bitcoin wallet consists of collection of UTXO i.e, Unspent Transaction present in wallet. We are not only concerned about balance and public key we are maintaining the transaction to track in-flow and out-flow. Blockchain always keep transaction history because for any new transaction we need to provide the transaction id (UTXO) from our wallet so we must save the transaction information