Bitcoin transaction's anonymity depends on public-private keys. The digital tie between a public and a private key is stronger than the tie between a person and his DNA. If you committed a crime but the police had your DNA you would hardly consider yourself safe. Furthermore there is a necessarily public list of transactions involving your public key and therefore connecting you to others by their public key. To call this anonymous seems a stretch!
Bitcoin is pseudonymous, not anonymous. Sure public keys and private keys are tied and all transactions are public, but unless you have revealed your real identity and connected it to your public key, people do not know who you actually are when you make a transaction.
Furthermore, you can generate as many public-private key pairs as you want and those are not actually tied to your identity.
So with Bitcoin, what you really have are pseudonyms, the public keys. You can have as many pseudonyms as you want and said pseudonyms are not necessarily tied to each other or tied to your real identity. Thus you are pseudonymous, not anonymous.