Addresses are textually encoded public key hashes, which means that somebody in possession of only the address cannot read the public key - but eventually the public key must be revealed in order to spend the funds which are locked by an address - so it may become a privacy concern if addresses are reused, which is discouraged anyway.
With a HD wallet, we have what are known as extended public keys, which, as well as a regular secp256k1 public key, also contain a chaining code and an index, and a fingerprint of the parent key. They usually begin with xpub. These should never be publicly shared as it is not necessary for anybody to know more than the secp256k1 public key in order to verify information from the blockchain. The purpose of the extended key is to allow you to create more child keys from a given key pair.
In the case of non-hardeded extended public keys, it is possible for anybody with the xpub to create the extended child public keys, which could reveal a lot more information linking your transaction activity. These kind of keys are useful for auditing purposes as you do not need to reveal private keys, but should not be shared otherwise.
Also note that, even if you never share extended public keys and you never reuse addresses, this does not give any guarantee of privacy. It may still be possible to link your transactions if multiple inputs are used, or if you transact with KYC services which relate them to other identity documents. You should never assume that any KYC service is not sharing that information with other parties, or that they won't at some point be hacked and have all of the data dumped on the web.
In regards to the way addresses are created from public keys, this is nicely illustrated by the diagram below, taken from the bitcoin wiki
