How does RBF (BIP 125) behave, when the unconfirmed UTXOs are being spent?

Question

Let's assume we broadcast transaction A (TXA), signalling Replace-By-Fee as defined in BIP 125. We pay 1 sat/vByte for the transaction.

One of the outputs goes to Bob. But Bob immediately spends his output, even before TXA is confirmed. Let's call this spending transaction TXB. TXB uses 100 sat/vByte.

Now, with still no transaction being confirmed, we replace TXA with a 3 sat/vByte fee transaction (let's call the replacement transaction TXA2). What happens in this case?

I can think of the following outcomes:

• Spending of unconfirmed RBF UTXOs is not allowed. TXB is rejected by the other nodes.
• TXA2 gets rejected, because there is already a spending transaction.
• TXA2 gets rejected, because the fees of TXA + TXB are higher (in total sat/vByte) than the fees of TXA2. But if TXA2 pays a higher sat/vByte than TXA + TXB, it replaces TXA, and TXB becomes invalid.
• TXA gets replaced with TXA2, TXB becomes invalid because the parent transaction has been replaced. The requirement is only that TXA2 pays more than TXA (in alignement with BIP 125 rules), what TXB pays in fees doesn't matter.

Answer 1

In order to be valid for replacement the challenger must have a higher feerate than (all) the conflicting transactions(s) and pay more absolute fees than (all) the conflicting transaction(s) and all its(/their) descendants.

In your example whether TXA2 will be accepted and replace TXA + TXB depends on the size of each (to deduce their absolute fee).