I think it would be helpful for you to identify your service here, so that I can encourage people to not do business with you.
Users generally have no idea if this will work safely or not, so a notice will not be sufficient. If you send users funds to addresses they did not supply you should be prepared to pay them out when they report the funds have gone missing or find yourself on the losing side of a lawsuit-- and there would be no shortages of experts willing to testify on behalf of someone you failed to pay due to this.
Yet each incoming transaction has its own list of incoming transactions with its own outputs sent to various "destination addresses", therefore the notion of their being no "from address" is not valid.
Sorry, that simply is not correct.
Coins have a history from which you could derive prior "To" addresses, that much is correct. But a prior "To" is not a "from". For example, if you ordered a produce from amazon, received it, then inspect the parcel with a magic wand that told you the last place the parcel was shipped you might find it was last sent to a reshipment center in Oregon. But that is just some shipper, not the source (amazon) and if you attempted to "return" the package by simply shipping it there it would likely end up lost.
Your order "from" amazon may have gone through dozens of steps or indirection before reaching you, and the last hop may have only the thinnest of relationships to the actual origin.
Even if the "from" is correct the user may simply no longer have access to the keys. With no more use for them they simply may have discarded them (e.g. a 'paper wallet' or other token). Users can setup persistent donation addresses intentionally taking due care to preserve the key material, and for donations the security/privacy implications are reduced because of the very nature of the payments (the donation addresses are public and the funds aren't provided in exchange for any consideration). And when access isn't lost completely, the keys in question may not be readily accessible, but stored in a safe deposit box or time lock safe. The funds could have also been spliced out of a payment channel and the prior-to could be inaccessible to the user without significant software modification or the help of a third party.
What privacy? Bitcoin doesn't provide any privacy...
Bitcoin does provide privacy and has since day one. It is specifically called out as a top level section in the Bitcoin whitepaper. The privacy Bitcoin provides isn't perfect for sure, but one of the biggest things that damages privacy is address reuse because it creates co-input linkage, causing otherwise completely independent spends to be obviously linked. Mixing "services" are frequently scams that steal user funds.
Your self-described "rant" seems to have also ignored the largest practical consideration: A great many wallets (perhaps the largest in total user count) are thoroughly incompatible with your intended behavior. If a user pays you from a hosted wallet and you "refund" to one of the apparent prior-to addresses, the funds will be delivered to some other random user (who will probably promptly withdraw their windfall, never to be heard from again).