The flaw in your thinking is that you assume your transactions will be confirmed. That is not the case. There is no guarantee that a transaction will ever be confirmed, and in fact, if the fee is too low, it is almost guaranteed that the transaction will never confirm.
You should care because the person you are sending Bitcoin to cares that it confirms. You should care because you won't receive what you are paying for until your transaction confirms. Most merchants will not consider the payment complete until at least one confirmation. If they do not consider the payment complete, you aren't going to be receiving any good or service that you have paid for.
Merchants (and other recipients) care about transactions confirming because it guarantees that the Bitcoin has actually been transferred. While a transaction is unconfirmed, it could be double spent and a different transaction that uses the same inputs and sends the Bitcoin elsewhere could be confirmed. If they accept the payment, provide the goods or service, and then find out that the transaction has been double spent, then they have both not made any money and lost the value of whatever has been provided to you. So receivers almost always wait for confirmations before considering a payment to be complete.