However, as far as I understand, new versions of Bitcoin will soon provide pruning capabilities, therefore allowing miners to only keep unspent outputs (without the entire chain) up to certain blocks, which drastically reduces the amount of space needed to verify transactions.
This has been the case since 0.8.0. The new addition is just to remove the requirement to keep block files around, which were only ever used for serving to other peers and for wallet rescans anyway. Incidentally the pruning function is incompatible with the current wallet, which would make P2Pool operation a bit difficult until it has been patched for operation with its own wallet system.
With all these new developments, wouldn't it make sense to embed P2Pool sort of technology inside Bitcoin Core itself, or are there bigger limitations that I'm not aware of?
It would have made sense in 2010, less so now. P2Pool has extremely high share difficulty which frustrates small miners, and it has all but lost all of the momentum it might have had a few years ago. Unfortunately it is also written in Python, which makes it a little lethargic and not suited for inclusion without a complete rewrite.
The mining landscape is also a lot different to even a year ago. Home mining is all but non existent, and most of the hashpower is concentrated under the control of as many people as would fit in a small elevator car. Those with miners will often have brands who have released hardware that actively damages P2Pool by discarding valid blocks (a sort of unintentional block withholding attack). Most mining hardware include small single board computers of some description which are underpowered even for stratum mining, which means the person needs to keep another full sized computer around to run a node on, which is apparently undesirable.
The budget for P2Pool development is almost zero as a result of the combination of all these factors. It would have been nice if this was a core part of Bitcoin to begin with, or something similar. Perhaps things would be different now if it had.
