I read briefly in a paper that there is some sort of transaction queue that the node keeps for each neighbor and that they will only select a random (?) amount of those transactions and send in an INV message to those nodes.
Note that this is specific to Bitcoin Core. Other full node software may not exhibit this behavior.
When Bitcoin Core receives a transaction, it adds the transaction to lists that it maintains for every other node it is connected to. Each node has its own list and that list contains all of the transactions that your node has received but may not have been received by the other node. After a certain random time delay, your node will then send messages to the other node. Among the messages it sends is an INV for transactions in the list for that node.
However not all transactions in the list are sent. Only some of them are sent. To select which transactions are sent, your node will sort the transaction list by the number of ancestors it has (so that parent transactions are relayed before child transactions) and by its feerate. Those transactions with the fewest ancestors and highest feerate will be at the top. Transactions are selected from this sorted list until either the INV message reaches its limit (which happens rarely) or there are no transactions remaining. Once transactions are selected, the INV is sent.
Why don't they send them all the transactions in the queue and why does it wait a random amount of time to send the transactions in the queue?
The random delay between sends is done for privacy reasons. The sorting of transactions is done to prioritize the transactions that will be selected first by miners. It also helps with privacy.
The reason these help with privacy is because a spy node that has connected to your node will not be able to always know what transactions your node has received and what your mempool looks like. It won't be able to know which transactions you have received first. Furthermore, due to the random delays, if a transaction originates from your node, it is possible that a node connected directly to yours receives your transaction from someone else first, thereby reducing the certainty of such analyses.