With assumevalid enabled, a full node skips signature validation (it's "script validation" rather than "signature validation" actually, but validating digital signature is what most scripts do), implicitly trusting the developers who hard-coded the default block hash.
However it's not bad, because the most tricky part, data availability, is still fully validated.
Sounds absurd to let everyone to download the whole years of transaction history? Developers just thought exactly the same as you. So they tried to make lightweight SPV client as safe/trustless as full node - the technique or idea to achieve this goal is called "fraud proofs".
Fraud proof was supposed to work a bit like "public opinion supervision", except that it's not "opinion", but "fact" based.
For example, if a transaction is spending a coin once again, Merkle proofs of those two spending transactions are the fraud proof. (then the corresponding block which included such double-spending transaction is also proven to be invalid) The proof itself is succinct, and verifiable without full blockchain.
Unfortunately, developers then given up on the idea of fraud proofs just because the data availability (edit: unavailability actually) can't be proven.
You can't know whether a piece of data is available without actually fully downloading it. Edit: You can't validate a piece of data without having it at your hands, therefore generating fraud proofs won't be possible. You are also unable to verify whether a piece of data is really missing without fully downloading it on your own.
Downloading the whole blockchain seems very wasteful especially when pruning is enabled. However, without downloading & validating the full blockchain ledger, you can never know whether something nasty, like inflation, or theft (confiscating coins without giving any valid signature), is hidden in some missing parts.
Even if assumevalid seems to skip some crucial validation steps, by fully downloading the whole blockchain ledger, you have already gained confidence to rule out the "hidden evil" problem above. Although there's still some trust (as mentioned in the beginning of this answer), but with the most tricky problem ruled out, it won't actually matter - real full validation doesn't cost much either, anyone can do it simply with assumevalid=0
.