My question is why transaction fees exist, even though the miners would still get rewarded with the newly created bitcoins when/if they mine a block. Miners will rather verify transactions with a higher transaction fee to eventually receive the fees themselves, but what if there were no fees involved and all the transactions would be verified without other considerations. The only reason I can think of is that with no transaction fees, when all the bitcoins are mined, there will be no more incentive for the miners and the network will fail. Is there another reason or is that it?
The block subsidy in Bitcoin serves as a decentral initial distribution mechanism and incentivizes the bootstrapping of the Bitcoin network. As you may know, the block subsidy halves every 210,000 blocks (approx. 4 years). Due to this reward schedule, more than 80% of all bitcoins have already been discovered, and in a decade, it'll be more than 95%. As the number of new bitcoins being added to the money supply dwindles, another mechanism to fund the security of the network will need to take over.
The source of that funding is meant to be transaction fees. This seems feasible, especially in light of us seeing the first block with transaction fees greater than the block subsidy last year.
The transition to a fee funded mining market is tightly coupled with the blocksize debate. For example, Princeton's Weinberg et al have shown that a fee-funded mining market may become unstable unless there is a constant backlog of transactions waiting to be confirmed. Some Bitcoin clients explicitly lock new transactions to the next block's height to encourage mining of a new block in contrast to miners attempting orphaning attacks on the current blockchain-tip for fee sniping.
The described dynamic is also cause for speculation that altcoins with oversized (and thus empty) blocks may need to transition to a constant block subsidy in the longterm to maintain a stable mining market.
Finally, the limited availability of blockspace and presence of transaction fees creates a fair market for users to signal the urgency of their transactions. This allows more valuable use of a limited resource to displace frivolous or inefficient use. The fee market especially discourages spam and business models that depend on abundant blockspace. Early acceptance of the inherent limitedness of blockspace is a prime driver for innovation and investment in sound second layer technology such as the Lightning Network.
People have been speculating for years, why Bitcoin was designed with the abrupt halvings of the block subsidy instead of a smoother payout or even a constant reward. There even was an attempt to turn the initial block subsidy of 50 bitcoins per block into a constant subsidy around the first halving, but the broader network rejected the miner's attempt to increase the money supply.
The reason for transaction fee is to pay miners when the block subsidy goes to zero. Since the block subsidy halves every 4 years (roughly), it will become negligible fairly soon, and transaction fees will thus need to make up for the lost block subsidy in order to continue to incentivize miners to continue to mine blocks.
Fees allow the network to prioritize transactions and avoid spam transactions which would fill the mempool. Additionally after the BTC reward will be gone Tx fees are the only way to incentivize the miners.