Really there are no bitcoins, just transactions. A transaction will have outputs of specific amounts, and if you have the key to spend a certain output but haven't spent it yet, it is known as your unspent transaction output (UTXO), and the amount of all your UTXOs is known as your balance. The private key is what allows you to spend a transaction output, it provides the proof you own it. And you don't send your private key when you make a new transaction, you simply sign a new transaction with your key to spend the UTXO you 'own', and create new outputs owned by the recipient, usually by assigning the outputs of the new transaction to their public key / bitcoin address.
The miners are also given bitcoins through special transactions known as coinbase transactions, which don't have inputs because they are creating brand new coins, but still have outputs to be spent by the miner.