This is a process known as consolidation - If your wallet has several small inputs, many wallets will often try to combine them into a single change input during regular transactions, especially ones made with a low sat/byte feerate. This allows you to spend the coins in a smaller transaction during possibly high traffic times, when the feerate may be much higher.
These days, many wallets only perform such consolidation if there are many outputs to a single address, as combining outputs from multiple addresses unnecessarily can lead to a loss of privacy. Unfortunately for you, it seems like in your case all the outputs do belong to a single address.
In the case of electrum, this doesn't appear to be a configurable option, although you could notice it is happening prior to submitting the transaction by using the "Preview" option - if you notice that the total fee or size or number of inputs consumed it too large, you can force electrum to build a transaction using only specific inputs by selecting them in your transaction history, then right clicking and selecting the spend from option.