Tx Malleability: Why is the Script not included in the Signed Hash?

Question

Sipa lists sources of tx malleability here: https://gist.github.com/sipa/8907691

It is clear to me how a change in the signature itself or the signature formating leads to a different tx hash (1. & 2.). But why is it allowed to modify the rest (may be "") of the scripts?

From what I understand only the txin script can be modified because the txout scripts are included in the signature hash(es). Obviously the signature itself can not be included in the signed hash.

For each signature the corresponding txin script is replaced by the appropriate previous output script (?) and all other txin scripts in the tx are removed.

Why is the txin script (without signature, so it is an empty string for a standard tx) not included in the hashed and signed data?

(Please correct me if I got something wrong.)

edit: trying to express myself more clearly:

Of course the range and format of the signature needs to be standardized. But there are other cases of malleability as Sipa points out like adding nonsense OPs to the tx_in script or using alternative OPs that have the same or a similar effect. I am only talking about these other cases.

My point is: The signature itself can not be signed but why are OP codes that may or may not be present in the txin script not signed?

Example:
Today:
normal txinscript: "sig" --> signature format: "" --> valid
malled txinscript: "OP_NULL sig" --> signature format: "" --> valid

Why not:
normal txinscript: "sig" --> signature format: "" --> valid
malled txinscript: "OP_NULL sig" --> signature format: "OP_NULL " --> FAIL

Signature format today replaces the whole txinscript with "". To me it seems that it would be safer to only remove the signature itself but keep the opcodes if present so that it is impossible to alter existing opcodes or add nonsense opcodes.

Is it because it is not know what part of the script is the signature? I think it could be marked as such should that be the issue.

Answer 1

You are absolutely right.

The only reason why this is done, is because that is not how the consensus rules currently work. Changing it would require a hard fork, updating every node and eventually every wallet.

Answer 2

The tx_in "script" is the signature (and pubkey and some formatting). It is a somewhat confusing name which kinda implies there is a tx_in script and then somewhere else these is this signature, but that isn't the case.

The signature can't be included in the input which forms the signature.

This diagram might help. https://en.bitcoin.it/w/images/en/e/e1/TxBinaryMap.png

The diagram on the left is a "standard" tx_in.

The tx_input structure consists of

Previous txout-hash (tx_id)
Previous Txout-index
Txin-script length
Txin-script 
sequence_no

However this is really just an abstraction the Txin_script is just the signature and pubkey which consists of:

L(sig)
0x30
L(rs)
0x02
L(r)
Sig r
0x02
L(s)
Sig s
0x01
L(key)
0x04
Key X
Key y (only for uncompressed keys)

So you saying the txin script is mutable is simply another way of saying the signature (and the formating). There isn't one single method to mutability. ECDSA itself has mutable signatures, the padding and formatting of the signature can be modified and still be valid.

Making tx_id immutable requires either computing the tx_id (hash) using an alternate method, where the payload of the hash is all immutable elements, or restricting the ECDSA signature to a particular subset of parameters, formatting, and padding such that for a given payload there is only one possible "correct" Bitcoin signature. Anything else while it may be a valid ECDSA signature would be considered invalid on the Bitcoin network.

It is a non-trivial task. Not impossible but it will not be resolved "fast and easy" and as far as I can see will require a hard fork of the network. This means additional time in testing, adding forward compatibility, and consensus building to avoid unintended consequences like leaving a significant portion of the network on the old fork.

Answer 3

My point is: The signature itself can not be signed but why are OP codes that may or may not be present in the txin script not signed?

The updated question provides a more specific context, and so I will answer it as a new answer.

There is no reason that Bitcoin couldn't be hard forked to accomplish what you indicate. If you are wondering why such a complex signing scheme? As Pieter says "ask Satoshi".

Even before we encountered these issues I always found the signature portion of the transaction to be overly complex and counterintuitive. If Bitcoin tx were signed like they are in just about any other cryptographic system the signature wouldn't be part of the input.

It could look something like this

tx header
input(s)
output(s)
signature(s)

The tx header, input(s), and output(s) would comprise the "tx body". The hash of the tx body is the tx_id.

To construct a transaction the client would create the tx body, compute the tx hash. The tx hash + priv key for each input would be used to generate a signature. The signatures would be appended to the tx body and the entire thing broadcast as the tx message.

To validate, remove the signatures from the tx and you are left with the tx body. Hash the remaining tx "body", verify signature. Verify the inputs, verify the outputs. Verify each of the signatures is correct.

Of course such a radical change to Bitcoin is still "possible" but it is unlikely it will ever be forked to accommodate that. Satoshi got a lot right but he wasn't perfect and this is one example.