A Transaction output has a field for amount and a locking script but no direct field for public address for which the output is meant.
This is not quite correct and seems to be the source of the confusion. A "locking script" contains the essential part of an address surrounded by the additional commands necessary to lock (or encumber) the bitcoins in that output to the address. Let's look at an example; first we get an address:
$ bitcoin-cli getnewaddress
Then let's ask the wallet what it knows about this address:
$ bitcoin-cli validateaddress 397yNmAFc1w6AK1gqkKNxJPb44jnvznm2b
Note the scriptPubKey---that's what you're calling the "locking script." Let's see what that script says:
$ bitcoin-cli decodescript a91451812710fc166bf61033afc0d2a8d7cda7a0fd8c87
"asm": "OP_HASH160 51812710fc166bf61033afc0d2a8d7cda7a0fd8c OP_EQUAL",
asm field is the interpreted form of the locking script;
OP_EQUAL are two parts of the P2SH-style encumbrance for addresses starting with a 3. The value in the middle
5181...fd8c is the hex form of the essential component of the address (other parts of the address are a version byte and a checksum; these aren't necessary to put in the locking script).
So, in short, the address
397yNmAFc1w6AK1gqkKNxJPb44jnvznm2b is a representation of the same thing as the scriptPubKey
a91451812710fc166bf61033afc0d2a8d7cda7a0fd8c87. The scriptPubKey is the data that actually gets placed in the output to indicate that a certain set of bitcoins are controlled by the owner of the corresponding address.
How does it find this subset of UTXOs which is locked to the public address?
A wallet that's part of a full node simply scans every transaction the full node receives to see if any of their outputs pays one of that wallet's scriptPubKeys (addresses). If it does, the wallet saves that UTXO in its own database until it's time to spend it.
Lightweight wallets typically ask full nodes or servers to help them find the outputs. Sometimes this entails scanning blocks basically the same way the full node wallet does with some extra steps (see BIP37), sometimes the server builds a giant database of every address every seen and performs lookup queries on it (see Electrum servers), and sometimes the full node builds a very compact index and sends that to the lightweight client so the client can request blocks that might contain its transactions (see BIP158).
Notably, wallets need to know which UTXO paid you in order to show you your balance, so wallets don't look up UTXOs when spending, they look them up when receiving.