I was wondering, can one use an OP_RETURN in the witness script?
Yes, you can, though I believe it is pointless. Executing an OP_RETURN opcode causes the script to return false immediately, marking the spend as invalid. As witness scripts are revealed only at spending time, this can only occur in an unexecuted branch (e.g. OP_0 OP_IF OP_RETURN OP_ELSE ... OP_ENDIF, where the OP_RETURN does occur in the script but is never hit). So I believe that any script with an OP_RETURN in it can be rewritten in a simpler/shorter way without it.
This is different from having an OP_RETURN in a scriptPubKey, as scriptPubKeys are revealed at creation time, not at spending time. A scriptPubKey of the form OP_RETURN <...> can never be spent (because spending it will cause the scriptPubKey to be executed, and thus fail), but it doesn't prevent the output from existing.
I looked at this file, and it wasn't quite clear to me how I might determine which opcodes are allowed in the witness script from this:
The script/script.{h,cpp} files only define the CScript data structure and some related functions. The validation rules regarding script execution are in script/interpreter.{h,cpp}.
I also looked through a couple BIPs, including BIP 143 and BIP 341, but they're kind of beyond my current skill level to understand how they might apply to my question.
BIP141 introduced segwit outputs, scriptPubKeys or P2SH redeemScripts of the form OP_n <witness program>, and permitted a new type of inputs (segwit inputs, which spend such outputs) which provide witness data not in the normal scriptSig, but in a new segregated witness section of the transaction data. As of today, there are 3 separate types of such outputs, each with their own validation rules:
- Also introduced in BIP141, P2WPKH outputs have a scriptPubKey or redeemScript of the form
OP_0 <20-byte hash>, where the hash is a public key hash. These outputs can be spent by revealing the full public key plus a signature with that key. No "real" script (with user-chosen opcodes) is ever revealed. The signature hash for these signatures is in BIP143.
- Also introduced in BIP141, P2WSH outputs have a scriptPubkey or redeemScript of the form
OP_0 <32-byte hash>, where the hash is a script hash. These outputs can be spent by revealing the full script (called the witness script), plus inputs to that witness script that make it evaluate to true. The semantics of scripts here only differs very slightly from the normal scriptPubKey script validation rules (which predate the BIP process), including for OP_RETURN. See the BIP under "New script semantics" for details.
- Introduced in BIP341 are taproot outputs; scriptPubKeys of the form
OP_1 <32-byte x-only pubkey>. Each of these can be spent in two different ways:
- Using a key-path spend (also introduced in BIP341). This involves just a signature (using a Schnorr signature instead of an ECDSA signature, defined in BIP340), and no script execution of any type is involved.
- Using a script-path spend (also introduced in BIP341). This involves revealing one of the scripts the taproot output committed to, together with a script version number. BIP341 however leaves the semantics of these scripts unspecified.
- BIP342 specifies the script semantics for taproot script-path spends with script version 0xc0, called tapscripts. They are mostly the same as the P2WSH script validation rules, but there are more differences than BIP141 had compared to legacy scripts. The major differences include BIP340 signatures, a new sighash scheme (mostly reusing the BIP341 one, but with a few extensions), the removal of
OP_CHECKMULTISIG and OP_CHECKMULTIVSIGERIFY, the introduction of OP_CHECKSIGADD, new resource limitations, and a few miscellaneous changes. OP_RETURN is not affected.
- taproot script path spends with a script version number other than 0xc0 don't have any script semantics associated with them (yet), and are anyone-can-spend (but nonstandard).
- scriptPubKeys and redeemScripts of the form
OP_n <witness program>, where n is between 1 and 16, excluding taproot outputs, don't have any script semantics (yet). They're anyone can spend, but non-standard.
Regarding terminology:
- The witness program is the 20-byte or 32-byte data element (pubkey hash, script hash, or x-only public key) pushed after OP_0 or OP_1 in the segwit scriptPubKey or redeemScript.
- The witness script is the script revealed in a P2WSH spend.
- The revealed script in a BIP342 spend is called the tapscript.
