I have created a transaction several days ago, with a very low fee. The transaction was meant to remove dust from my wallet, so increasing the fee won't help as the transaction is very big (1.6KB), and I can't afford to spend more than a few (28 right now) satoshis per byte. Now as expected, this transaction is stuck forever. I now want to send some of the dust somewhere else. If I understand correctly, I could just create a new transaction that uses the same (not transferred yet) bitcoins, which would then automatically invalidate the waiting transaction. How can I do this with Electrum? If I just create another transaction, it will wait for the parent to confirm.
You need to perform a replace by fee transaction
Go to preferences and activate dynamic fees if that's not already set. You can then right click on a "low fee" tx in the history and select "Child pays for parent". You then have a fee slider with "within 2 blocks" pre-selected from a dynamic fee. You will now see a preview window for the transaction. Click sign and broadcast the signed transaction and the fee is bumped.
This will only work if you have change from the transaction to your own wallet (usually the case). Otherwise you can ask the recipient to do a CPFP on the transaction. The CPFP transactions in Electrum are by itself RBF, so you can further bump the fee after doing CPFP if needed (which means you could select a lower target like 25 blocks and bump further from there if you want to save on fees).
Make sure you select a server that runs Bitcoin core (the servers usually print what they are running in the console tab) because BU won't accept RBF and will prompt an error (you can still switch then). Most Electrum servers are running core though (I'd guess ~80%+), so it'll work out most of the times.