The Bitcoin network uses the blockchain to form consensus on the journal of transactions. The blocks commit the network to a specific order of events and guarantee propagation. Blocks are found roughly every ten minutes and are limited to 4,000,000 weight units (or 1,000,000 vbytes). Said blockweight limit and cadence of blocks are what limits network's transaction confirmation rate: with about 2,000 to 2,500 transactions per block every 600 seconds, the transaction rate is currently closer to 3.5 tps in practice. Note that a transaction may perform more than one payment.
Before transactions are organized into blocks by miners, the unconfirmed transactions are tracked in the memory pool (mempool) of each node. A node's mempool can hold about half a day's worth of transactions by default. Miners generally pick the transactions with the highest feerate first to build their block to optimize their revenue. When peak demand subsides, the network eventually chews through the backlog, also confirming transactions with lower feerates. Altogether, this forms a market for blockspace in which users can bid with transaction fees to signal the urgency of their payments.
Indeed, how many actual payments would it take a (good or bad) actor to fill the network (or would the payments simply get all grouped together infinitely), is the limit perhaps 3.5*4million per sec?
As described above, miners generally pick the transactions with the highest feerates to form their blocks. If the attacker wants to monopolize all blockspace, they'll have to continuously displace all other transactions from the top of the queue. However, other users are free to resubmit their transactions with an increased feerate to vie for the top of the queue. This would either break the attack or quickly drive up the cost, as all other users acting independently would outspend a low feerate spam attack.
In the 2017 bubble, we saw over a dozen organic blocks with more than 12 BTC in fees. Assuming that an attacker would need to pay a corresponding ~1,200 sat/vB to monopolize blockspace, their attack would cost about 72 BTC per hour.