In theory, you pay whatever fee you want, in order to bid for space in the blockchain. Miners are free to include or ignore your transaction, but a higher fee likely incentivizes them to accept it.
In practice, what matters is the fee per vbyte (feerate) of the transaction (vbyte is defined as the size in bytes of the entire transaction, but every byte in the witness only counts as 0.25). The reason is that the actual protocol rule says that blocks are limited to 1000000 vbytes. That is the constrained resource. Miners want to make as much money with the space available, and thus will select transactions with the higher fee/vbyte (feerate) first.
Adding more inputs does increase a transaction's vsize, but so adding more outputs, or using more complex scripts. It's not a simple function of the number of inputs (or outputs). For example, spending a multisig output generally takes up more vbytes.