Short answer is yes but you shouldn't, see point 1 below, and long answer is no its not safe or advisable.
I would say what you are proposing which is akin to a brain-wallet style single private/public key pair in this question is inadvisable for a few reasons:
1) The biggest risk is a lack of return change address (due to the absence of BIP32) in the case of sending anything less than the full balance will result in 100% loss of the difference (see here for more )
2) because of the checksum used in BIP39 which would alter your pasted binary data (i.e. pasting 256 bits for 24 words will later give you back 264 bits because of the 8-bit checksum that is appended to the last 11-bit group).
2) Besides Electrum, most wallets only allow the importing of mnemonic recovery phrases that are BIP39 checksum compliant and won't allow you to use an elliptic curve private key. In my opinion, it is better to have a checksum compliant mnemonic, despite its security being 256 bits, or 8 bits lower than 264-bit mnemonic with no valid checksum. Because it is more widely usable across wallet, in case electrum goes down or no longer works/supported, etc...
3) Decoding risks: if you are using a software that generates the private key in Base58 format, there is potential payload/checksum data attached that will increase the risk for errors when converting back to hex manually. Otherwise, if your are obtaining the private key in hex format, and are absolutely sure that it is the correct one in terms of the elliptical curve calculation to derive the public key and subsequent bitcoin formatted address, you can backup that string of hex characters in a number of ways.
Still, it would be better to back up the entire bip 39 Mnemonic crypto vault and then use an offline tool to access any related needed private key such as Ian Coleman's BIP39 tool offline (note: I am a contributor to that Github repo). Although you will still have to deal with converting Base58 encoded private keys, which I don't reccomend unless you know 100% how to do so without error.
4) Error checking: not all 256-bit numbers are valid private keys, as per the Elliptic Curve specifications used in Bitcoin (here is a related applicable post). So there must be error checking if you are generating the private key yourself manually, to make sure the key is smaller than the maximum permitted value 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141.
5)Regarding potential collision of private keys with mnemonics: yes there such collision, and it would be equal to two people randomly choosing or guessing the same private key (i.e., if the initial entropy for a mnemonic is the same entropy for an elliptic curve private key). Otherwise, the keyspace of private keys is the referenced above in point 4, and no two private keys point to the same address on the elliptic curve secp256k1 used in Bitcoin. The initial entropy range is 2^256 which is equal to the range of potential mnemonics 2048^24 minus the mnemonics with invalid checksums (otherwise the keyspace of mnemonics would be 2^264 ==2048^12 without the checksum requirement but both still wouldn't change the curve private keyspace ).
Again, I think it would be safer to follow the industry standard and use an HD wallet (which you would backup with a mnemonic).
P.S. You can further compress your backup using just the first 4 letters of each of the 24 words, provided that you are using the English BIP39 version, and concatentate them as a string of 44 characters. But when importing into a wallet, some don't autocomplete and you may have to lookup words manually from the list of 2048 BIP39 English words. I use an ecoding scheme that can further reduce the 44 charcters to 22 characters (including special symbols and numbers) with no information loss, but only to complement backups and not as a standalone replacement for BIP39 mnemonics.
