This is approximately how Bitcoin full nodes already work, though with one important change.
Bitcoin, at the protocol level, has no notion of addresses, and doesn't work in an "account" based fashion, for privacy reasons. Instead, it uses a "UTXO model" (unspent transaction output). The idea that the state of ledger is a set of UTXOs, which you can think of as "coins". Every transaction consumes certain specific, referenced coins in their entirety, and then creates new coins whose combined value does not exceed the consumed coins' value. Each coin has an identifier, an amount, and a locking script which determines the conditions under which it can be spent (as a simplification, you can think of locking scripts and addresses as the same thing). However, coins all have their own identity, and are not identified by their script/address. In order to create a transaction, you have to specify exactly which coins you're spending. The address just controls what signature is expected to sign for it, but there is no balance per address at the protocol level, just coins.
The reason for this UTXO model is privacy: account based models (where the rule is that sending accounts have sufficient funds available) encourage reuse, because reusing the same account is cheaper (both economically and computationally) than creating new accounts for each receipt. Such reuse is horrible for privacy, as it makes it trivial to link transactions of the same user together. In general, we want a system where acting in a more private manner is not more expensive than not doing so. In a UTXO model, reusing addresses is just as costly as using a new address for each receipt, satisfying that property (there are of course other privacy leaks too, not covered by address reuse).
So, with that out of the way, the equivalent of the "address: balance" list you imagine, is actually a "coin: value, script" list, and it is called the "UTXO set".
Bitcoin full nodes actually do not use the blockchain for anything during validation. They download all blocks, check them against the UTXO set (e.g. checking that all coins consumed by the transactions in the block still exist), then update the UTXO set (removing spent coins, and adding newly created coins), and then move on to the next block. Some nodes do keep the block after processing around, but it's just so that they can provide to other nodes who ask for it; they don't use it for validation of future blocks, that just needs the UTXO set. So-called pruned nodes don't keep blocks around after validation.
The current UTXO set is a few GB when stored on disk, and that is the minimum needed to run a node. It is far lower than storing the whole blockchain, which is a few hundred GB.
Now, in order to fully validate without trusting their peers, nodes do need to download and process all blocks, so that they can build up the UTXO set. It's in theory possible to just get the UTXO set from another node, but that does involve trusting that that UTXO set is correct. There is no P2P protocol for doing so, though there is ongoing work (called "assumeutxo") to permit something along these lines.
