Note that an address is just an encoding of some data and that not all addresses have to be the encoding of hashes.
The reason we use an address is because it is easier for humans to read, identify, and transport. But I suspect you are asking why we use the hash of public keys rather than the public key itself.
Encoding the hash of a public key as an address began with Satoshi writing the original Bitcoin client. It is likely that he chose to encode the hash rather than the public key itself in order to be more space efficient - it is easier to transcribe an encoding of the shorter hash rather than an encoding of the public key itself. Since Base58 includes both upper and lower case characters as well as having only a checksum at the end, it is annoying to read out or copy it easily, so shorter is better.
Since then, new address types that have been introduced have followed the same practice pretty much because addresses were first done this way, so why change. When P2SH addresses were introduced, it made sense (and still makes sense) to encode the hash of the script rather than the entire script itself since scripts are variable length and can be very long.
However the proposed Segwit v1 addresses for taproot will not be using hashed public keys. Rather the it uses public keys themselves encoded as bech32 addresses. Even though this is not as space efficient, it does make transactions more efficient, and does away with the suggestion that outputs might be quantum safe. Furthermore, because it uses bech32, the case-ness of each letter does not matter and since it uses BCH error correcting codes, it can tell you where you've made an error and what that letter should have been (assuming that you have not made a lot of errors). So this avoids the other issues with using Base58 encoded addresses.