Does Rootstock need segregated witness or any other bitcoin improvements not currently included for it to work?
- No, Rootstock does not need segregated witness.
- Yes, Rootstock does need another Bitcoin improvement not currently available on mainnet.
Rootstock & segregated witness
Segregated witness (segwit) eliminates all known third-party transaction malleability, which is an extremely useful feature but something that's only required in cases where a transaction has to be created before all of its parent transactions have confirmed.
For example, in a Spillman-style payment channel, the refund transaction is created and signed before the deposit transaction is broadcast---which is dangerous because if the deposit transaction is mutated (malleated), the refund transaction becomes invalid.
However, for a two-way pegged merged-mined sidechain like Rootstock is planning to use, the design makes the peg transactions robust against double spending (by waiting for many confirmations) in a way that also makes them robust against malleability.
Segwit also increases the maximum possible transaction size, which can make it more a bit more affordable to use sidechains, but not in any significant way and it isn't a requirement.
Rootstock & other Bitcoin improvements
Rootstock wants to use a two-way-peg merge-mined sidechain (also called a decentralized sidechain). There is no current know way to do this with just the opcodes currently enabled on Bitcoin mainnet, so this is not possible today.
Broadly speaking, there are a few different proposed approaches to enable this:
Extend Script's flexibility: add one or more generic opcodes to the Bitcoin Script language to allow it to validate what are called SPV proofs or merkle proofs---the kind of data that SPV clients use today to determine if a transaction is part of a chain of block headers.
Add a special SPV proof opcode: give Bitcoin Script the ability to directly validate SPV proofs. This would make the sidechain contracts simpler and cheaper, but to some engineers it feels a bit weird to add opcodes specialized for one purpose.
Add a special reverse peg opcode (drivechain): give Bitcoin Script a special opcode that says, "I trust miners". Anyone who pays this opcode allows miners to spend those funds however they would like; then arrangements could be made with miners to only spend that money if they followed the sidechains rules.
At least one of the approaches above would need to be enabled for Rootstock to follow their plan; Rootstock's current preference is for #3, drivechains.
It's worth noting that if Rootstock wanted to use federated-pegged (fedpeg) sidechains, such as those implemented in the Elements Project, then no changes would be required to Bitcoin today. However, fedpeg sidechains rely on trusted signers for every block and so are not decentralized.