I understand a "wallet" to be a private key and a collection of associated public keys. But if I was to be given only a private key, would I be able to determine if it had any associated addresses/public keys that had unspent bitcoin in them? If so, how?
A Bitcoin private key always have exactly one corresponding public key from which the Bitcoin address can be derived. The derivation paths looks as this: Private key -> public key -> Bitcoin address. If you e.g. only have the public key you can determine the bitcoin address, but not the private key. Given only the Bitcoin address you can in fact not even determine the underlying public key.
Therefore all you actually need to save is the private key, from which you can find the public key/address and check if there are any unspent outputs for it.
On top of that deterministic Bitcoin wallets have a clever way to derive determanisticly a nearly infinite amount of private keys, so all you then would need to keep safe is the seed for this derivation.