Transaction fees pay for two completely different things which should not be confused. One is the marginal cost of verifying, propagating and storing the transaction; the other is the amortized cost of hashing to secure the network.
Addressing the first need is relatively easy (though not yet solved because the receiver of the reward is not identical to those who have to pay the cost); a miner will not include a transaction if its fee is below the marginal cost. Hence senders who want their transaction included will pay at least the marginal cost (as will be determined by historic equilibrium).
The second need is more subtle. For the sake of discussion we can assume the marginal cost to be 0. Then a miner will always want to include any positive-fee transaction, however low, since it costs him nothing and he gets the fee. But if everyone does this, tx fees will race to the bottom, the total profit to be made from mining will shrink, the network hashrate will decrease and it will be vulnerable to attack.
This is a bargaining game between miners and senders, where "miners" are not a monolithic entity and suffers from a tragedy of the commons; the self-interest of each miner harms the totality of miners, and hence, the network security.
The current protocol enforces a data size limit of 1 MB per block. This makes space in a block a scarce resource over which transactions will compete, increasing the equilibrium fee. However, this is hardly efficient or optimal. The size limit likely will need to be relaxed at some point.
When this happens, I believe the optimal way to give miners the bargaining power they need is to enforce on the protocol level a hard limit on the total value of transactions per block. This will make miners choose transactions based on their fee-to-value ration; thus making sure that those sending high-value transactions, who gain the most from the protection of the hashing network, pay for the privilege accordingly.