What are the relevant security concerns here? Is it hardened or non hardened derivation (or both) that are relevant here? Is BIP32 key derivation one of the reasons why Bitcoin design decisions are often different to those made in research papers without a focus on Bitcoin?
Alternative answers as always are welcome.
BIP340 does actually (at least partially) answer this.
Key prefixes were included in the design of BIP340 Schnorr signatures as without them "related key attacks" (e.g. attacks with BIP32 unhardened derived keys) are possible. Related key attacks are where you are able to produce a signature for a BIP32 unhardened derived child key without access to that key if you have the signature of that same message from the parent key.
a third party can convert a signature (R, s) for public key P into a signature (
R, s + a⋅hash(R || m)) for public key
P + a⋅Gand the same message m, for any given additive tweak a to the signing key. This would render signatures insecure when keys are generated using BIP32's unhardened derivation and other methods that rely on additive tweaks to existing keys such as Taproot.
Key prefixes are defined in BIP340 as:
...we choose key prefixed Schnorr signatures which means that the public key is prefixed to the message in the challenge hash input. This changes the equation to:
s⋅G = R + hash(R || P || m)⋅P.
It can be shown that key prefixing protects against related-key attacks with additive tweaks. In general, key prefixing increases robustness in multi-user settings, e.g., it seems to be a requirement for proving the MuSig multisignature scheme secure (see Applications below).
BIP340 also links to this paper (On the Security of the Schnorr Signature Scheme and DSA against Related-Key Attacks).
Is it hardened or non hardened derivation (or both) that are relevant here?
As far as I can work out hardened BIP32 derivation isn't affected by related key attacks or any other security concerns so we are only concerned with unhardened BIP32 derivation when designing secure key aggregation multisig and threshold schemes like MuSig(2) and FROST. The design of these schemes are obviously constrained by the design decisions in BIP340 as they need to generate valid BIP340 public keys and be able to produce valid BIP340 signatures under the rules of SegWit version 1.