If you don't validate a transaction, then you are at risk for relying on it, propagating it, or including it in a block you're mining. For this reason, the default behavior is to validate every transaction (I think!). Otherwise there is some reliance on free transaction validation, which turns out not to be free because of the risks. There is currently no simple way to pay for transaction validation in order to mitigate those risks. I think there should be.
Some guys at Microsoft did some research into the idea of incentivizing transaction propagation (which would necessarily require validation for those getting paid).
I am wondering if it is worth it to develop an idea I had and present it to the bitcoin dev list. It's called "bonded proof" and consists of a proof message which is a valid block (except that it's hash isn't low enough) and the TXOut that is the bond. It isn't actual proof, but tentative proof that can be verified and, if found to be wrong, redeemed. The bonded proof is signed by the bitcoin address that holds the bond. I imagine the TxOut would have to hold at least 100X a standard Tx Fee, but since it's not expected to ever be redeemed, it could hold far more just as a show of good faith.
If the proof is wrong, the maker of such a bonded proof can expect the bonded proof to be redeemed in a block. The "transaction" that redeems such a failed bonded proof would need to include the entire proof message, plus an index into the block to identify the invalid transaction, plus a btc ("disproof") address to receive some part of the bond (say 50%). Block validation would then have to verify that the identified transaction was actually invalid, and this would cause (say 50% of) the TxOut holding the bond to be transferred in the coinbase transaction, while the rest would go to the disproof address.
Miners would be free to ignore any such bonded proof, or to use them if and when they appear to be valuable, but I propose the following mechanism as a default: The maker of a bonded proof ("weak") block would request a fee by making it the coinbase transaction in the weak block. Miners would add a transaction sending that fee to the address in the weak coinbase transaction if they want to encourage the block assembler. By default, any time a node has a brand new transaction to broadcast, it would make a weak block with the new transaction in it as long as the operator has specified a TxOut for the bond.
Selfish miners could turn this off if they wish to cheat those who assemble these blocks, just as they are already doing (probably because it's difficult not to) to those who propagate transactions. I suspect behavior would be better than that. Thousands of people have run full nodes without compensation, and this just makes it easier to reward them, and offload to them the time-consuming validation step, which they are doing anyway (by default).
Furthermore, non-mining nodes would have an incentive to validate these weak blocks in order to find an invalid transaction (root out cheaters who don't bother validating) and redeem the proof block.
The idea of bonded proof appeals to me not just in the case of weak block validation, but as a general mechanism. It isn't called that with regards to the Lightning Network, but the idea is very similar: motivate helpful behavior while requiring that the helper stake some bitcoin on the claim that he is helping.
the default behavior is to validate every transaction (I think!)
: with BitcoinCore, yes, but not "client-only" (SPV)