How is the fee specified?
The transaction fee is paid by the sender.
Bitcoin transactions consist of three parts. The inputs specify which unspent transaction outputs (UTXOs) get consumed and satisfy the corresponding locking scripts, the outputs define which new UTXOs are created by the transaction per amounts and addresses, and the header keeps track of metadata like the version field, the locktime, the input and output counts.
The transaction fee is only specified implicitly, by the difference of the input amounts and output amounts:
transaction fee = Σ(inputs) – Σ(outputs)
How do miners collect fees?
When a miner finds a valid block, they pay themselves the block reward. The block reward consists of the block subsidy (new coins), and the fees of the transactions included in the block.
When Bitcoin users submit transactions, the nodes on the network relay the transactions and collect them in their mempools, their queue of unconfirmed transactions. Miners select transactions for their block templates from their nodes' mempools. A block is limited to 4,000,000 weight units, which translates to about 2,500 to 4,000 transactions, depending on the transactions' weights. Given the limit, miners prefer transactions higher feerates (fee per weight), since that will yield the greatest block reward.
How do users pick fees to get confirmed?
When users send transactions, they must consider how urgently they want their transactions to be confirmed. Depending on their choice, their wallet software estimates a feerate. As their wallet software picks a set of inputs to fund the transaction, the wallet keeps track of the weight the inputs and outputs will incur, and allocates the absolute fee that results in the targeted feerate. As the transaction then queues, this fee functions as a bid for the necessary blockspace to include the transaction.
You may have noticed that the fee is not related to the payment's value at all. A transaction with one input and one output will cost the same whether you send $10 or $10M. The fee is purely a function of the weight of the transaction and the urgency of the user. This results in larger payments having a lower relative cost and Bitcoin's base layer being poorly suited for micropayments. The slow confirmations, high cost for micropayments, and scalability concerns provide some of the motivations for the development of the Lightning Network, a second-layer instant payment system on-top of Bitcoin's base layer.
A main challenge for feerate estimation is that it's unpredictable when the next block will be found and how many transactions will get queued until then. The probabilistic nature of mining and varying demand causes interesting dynamics for the global transaction queue. Some of these problems can be sidestepped by issuing replaceable transactions which can be reprioritized by overwriting them with transactions of bidding a higher fee. Note that most nodes will not relay transactions sent with feerates of less than 1 sat/vB.