When a new full node joins the network, does he, by default, compute the UTXO set from scratch from the blockchain so far; or does he simply trust the UTXO set he received is valid?
The UTXO set cannot be validated, except by going through all blocks in history.
The P2P protocol does not even have a way to stream the UTXO set from one node to another, so yes, full nodes will always download and verify full blocks, and rebuild the UTXO set from it.
There are various ideas (some of which are very old) to make blocks commit (= contain a hash) to the UTXO set, or a related data structure. That way one could synchronize the UTXO set for a state in the past (which means a trust in miners), and fully verify going forward from there.