Once spent, the wallet removes the UTXO from its cache of unspent transactions. The transaction is not removed from the general database (the blockchain or block index). Full nodes running with txindex=1 keep every transaction forever. Some wallets though optimize the database to recover disk space - but this is a wallet-specific optimization and not part of how Bitcoin works.
For instance, Bitcoin Core has many caches and optimizations. One of them is the UTXO cache. The reason to have this cache is kind of obvious: if you had to search the entire blockchain for your own unspent coins before every transaction, that could potentially take hours. So Bitcoin Core keeps track of your UTXO's. Your UTXO's output amounts add up and become the balance you see on the wallet software.
The UTXO cache never points to previous transactions - it's just an optimization cache that makes it easy for Bitcoin Core to find TX's. The algorithm which builds the UTXO cache knows which transaction the UTXO came from, but the cache never points anywhere except to the TX index where the unspent output is located.
The pointer to a previous transaction only exists when you spend a UTXO and then a new input TX is created on that transaction. That input will point to the previously unspent transaction and that former UTXO will become spent. A new UTXO is then created after this transaction is mined. This UTXO becomes part of the Bitcoin balance of the recipient, just like the UTXO was part of your balance before you spent it.
This image (courtesy of bitcoin.org) might help illustrate how it works:

See those UTXO's dangling without a new TX ahead of them? Until they're spent, they remain as UTXO.
The arrows in that illustration show the money flow direction, but technically the arrows point the other way around : the inputs point back at some previous UTXO output. Once an input points at an output that output is no longer a UTXO, it is a committed TX. The input links the previous TX to a new UTXO and the previous one is marked spent. Bitcoin Core then removes this previous UTXO from the cache.