A simple Bitcoin wallet consists on one of more pairs of public and private keys (I'm saying simple here as some wallet structure allow for deterministic public key generations and private keys that can only spend part of the wallet, but the principle with public/private keys remains the same). The "link" to Bitcoin initially happens during block generation where a certain amount of Bitcoin is generated and sent to the miner's public address; this is merely a record in the blockchain. Then these coins can be sent to other public addresses using transactions.
The most simple transaction has one input and one output; i.e. it spends coins from one source address to one destination address. To be valid, the amount of output coins must not exceed the amount of input coins, and to be verified the output address hash is signed by the input address' private key.
In real world though, wallets balances become the sum of all inputs to that wallet, so when you make a transaction the software will select the desired number of inputs to get a sum equal of greater to the desired transaction output + fee. If the amount exceeds what you want to spend, the remainder can be send back to one of the input address or even a new address of your wallet, so you basically send yourself the exceeding amount of Bitcoins. Any unspent amount in the transaction is considered fee and is collected by the miner generating the block.
Transactions can also be much more complex; Bitcoin allows for a very powerful scripting system that can make transactions spendable in many ways. You can read more about bitcoin transactions and scripting system here:
https://en.bitcoin.it/wiki/Transactions
https://en.bitcoin.it/wiki/Script
When you send out a transaction, it is relayed to the network for inclusion in a block if it passes common rules for transactions (validity, fee, size, etc.). A miner will eventually pick your transaction and include it a block (the transactions are actually included in a merkle tree whose root hash is in in the block header - the block header hash is the proof of work that has to match the difficulty requirement for the block).