what is the process that is actually happening under the hood when I click the 'transfer Bitcoin' button of my wallet in terms of the keys?
Your wallet selects from available coins (UTXOs for which your wallet knows the private key) a sufficient number to cover the amount being "sent" plus the selected transaction-fee.
These selected UTXOs form the input side of the transaction the wallet constructs. The output side has at least one output that contains a locking script derived from the "address" provided by the payee. The locking script is sometimes called a scriptpubkey for historical reasons.
For each coin being spent, your wallet then creates a cryptographic signature of most of the transaction data using the corresponding private key and the ECDSA algorithm.
This signature data is put into the unlocking script for each UTXO. This unlocking script is sometimes called the scriptsig. It proves to the rest of the nodes in the network that you have the right to spend those UTXOs. The exact contents needed in the unlocking script obviously depend on the specific type of locking script in the prior transactions in which those coins were sent to you (i.e. came under your control)
The unlocking scripts are part of the created transaction but obviously cannot be part of the transaction data that is signed because their contents are not known until after the signature has been created.
The above applies to the simplest types of transactions. There are complicated types of transactions, such as n-of-m multisig, which I have not attempted to describe here.
How does the bitcoin network use public [...] keys [...]
In at least the simplest forms of transactions, nodes in the Bitcoin network use the public key provided to verify that the cryptographic signature in a transaction is a correct cryptographic signature of the transaction data.
Putting it very loosely:
bigint signature = ECDSA.sign(data, private-key)
boolean validity = ECDSA.verify(data, signature, public-key)
Things must get more complicated at a detailed level when you compare P2PK vs P2PKH let alone P2SH, segwit outputs and multisig transactions.