To claim (spend) the bitcoin from an address one needs to create a transaction and provide two things:
- a public key (from which the address you are trying to access is actually derived from**) and
- a signature of the transaction (with the private key that corresponds to the above public key**)
When you spent some money from an address the rest will typically go back to the originating address as change (see section Output here).
You are correct that the public key will be exposed after the first transaction. An attacker would have your public key. However, this poses no threat since the public key can be, well, public without compromising security. This is how public key cryptography works.
It is the signature that provides the real security. To create such a signature one needs the private key corresponding to the public key. And that is never revealed in the blockchain.
The above two links provide a lot of detail of how public key cryptography and bitcoin transaction work.
** Note that a bitcoin address is created from the public key with a one-way function (the reverse is not possible). Similarly, a public key is created from the private key with a one-way function. i.e. with the private key one can re-create the rest.