I'm looking to track transactions in a non-malleable way. To achieve this I've read the recommendations in this answer, but I don't understand how the order of transaction inputs isn't malleable. It's my understanding that each separate input is signed by the private key it's address is associated with, and thus a malleability attack could just swap the order of inputs and a tracking hash dependent a specific order would be rendered useless. Am I missing something here?
The signature associated with an input doesn't just sign that input, it signs the entire transaction payload - basically every part of the transaction except the signatures themselves. (Technically, it appears to actually sign a hash of a serialized version of the transaction.)
It has to be that way. Otherwise, you could take someone else's signed transaction and modify the outputs to pay it to yourself instead of the intended recipient.
So if you modify the order of the inputs, you modify the signed payload. The signatures will no longer verify and the modified transaction will not be accepted as valid by anyone.
The malleable transaction attack worked by modifying the signature itself to a different but still valid alternate version. The payload of the transaction never changed.
I have a feeling you are missing the index.
Each input is actually a transaction output from a previous transaction. The whole previous transaction is identified by the transaction hash, however the previous transaction can be paying to one or more addresses. Usually you control at least one of those addresses and so when you make a payment your software is
- identifying the (hopefully already confirmed) transaction
- identifying the index of the payment (ie your address) which you will use coins from
Point 2 is not malleable, the index is specific to the previous transaction. However you are able to compile your payment transaction in any order you choose, before it gets signed, and becomes your output.
This has nothing to do with the transaction malleability issue though.