Why does Bitcoin build in transaction fees before it hits the maximum allowed amount of coins? That is, shouldn't the reward of coins for mining a block be enough to keep the system going? In addition to the trouble we're seeing now (a high value of Bitcoin makes doing simple transactions uneconomical), it seems like building in a transaction fee that the user can select would rig the system such that higher fee transactions would get prioritized over lower-fee ones, which would restrict some people from using Bitcoin. I guess all cryptocurrencies do this so there is no way of avoiding it.
One reason behind a transactions fees is to make spam attacks expensive. Without fees, a malicious party could create a lot of no-fee transactions for a DoS attack on blockspace. And without fees a miner wouldn't be incentivised to include the highest fee transactions in a block. He could attack the network by mining empty blocks or including spam transactions. (e.g. transactions with clearly and solely exist to bloat the UTXO-set or to fill up blocks).
And correct - with a high enough transaction fee some people in e.g. third world country's can't pay for the transaction. However enabling users to select a transaction fee is necessary. A user that doesn't need fast confirmations can pay a lower fee than a user with a need for an instant confirmation. Thus a fee-market can develop.
This might be an interesting read regarding this topic: The Fee Market Explained by Jimmy Song
This is not a question specifically about transaction fees; you could just as well ask "Why do we have to pay for oranges?" and it would be essentially the same question. At a price of zero, the quantity of confirmations (or oranges) demanded would exceed the quantity supplied. Now how do you decide which people get their transactions confirmed (or which people get to eat the oranges)?
First, there needs to be an incentive to include any transactions in a block. While mining rewards are enough of an incentive, a healthy fee market ensures that miners attempt to add as many transactions as they can to a block.
Second, there is limited block space. How much space there should be is a huge point of contention, but pretty much everyone agrees that blocks need to have a reasonable limit in order to keep participation in the system accessible, and thus keep the system decentralized. As such, transactions will have to compete for space in a block. This is decided by what we call the "fee market", where transactions will have higher and lower fees based on the supply (amount of block space) and demand (number of transactions) of the network.
This means that, yes, transactions with higher fees are prioritized by miners (the ones collecting the fees) over transactions with lower fees. That's how a free market works.