Let's think of blocks as a basket. Each transaction is an apple weighing 0.1 kg. You are the network. Currently, let's say we have a basket large enough to carry 1000 apples (10kg). As the network, it takes you around 100 seconds to carry this from point A to B.
Our throughput is 10kg/100seconds = 0.1 kg per second
.
Now, we want to increase this.
Our options are to either reduce the amount of time it takes to carry the basket, or increase the size of the basket.
Let's increase the size of the basket first. Now, we can fit 2000 apples (20 kg). However, because it is now heavier, it takes us longer than 100 seconds to carry it from point A to B. With a linear scale, we would expect it to now take twice as long, or 200 seconds. Thus, this doesn't actually increase throughput, since 20kg/200seconds = 0.1 kg per second
What if we try to change the time instead?
As we saw, the time increases when capacity (weight) increases. Thus it should decrease when capacity decreases, but that would essentially have the same effect as we saw with the capacity increase.
However, what if we simply reduced the distance from point A to B? If it's only half the distance, we can carry the same capacity in half the time. Thus, 10kg/50seconds = 0.2 kg per second
. We doubled our throughput! Unfortunately, this comes at the cost that our network's reach has been halved.
In the case of Bitcoin and other cryptocurrencies, the same problem can be restated. The distance A to B is the peer-to-peer network. The capacity (or size) of the basket is the blocksize, and apples are transaction.
A good network is one where we have an optimal block size for containing as many transaction as possible, while still being able to broadcast that block efficiently to the rest of the network. Since mining depends on the miners being able to build on top of the latest block, it is very important that participants are notified of the latest block as soon as possible.
If we have 100 MB blocks, it would take a very long time to broadcast this to enough of the network. This would mean that we have to increase the blocktime sufficiently, so that mining can still take place fairly.
Similarly, going in reverse, if we were to reduce the block time, we would also need to reduce the block size so that blocks are still broadcasted quickly enough to work with the faster block time.
Bitcoin is not the best example of this, due to a relatively low block size with a relatively high block time. It would be feasible to increase the block limit without requiring adjustment on the other factors (indeed, this is essentially what segwit did in a slightly roundabout manner).
A better example may be Ethereum. There is no fixed block size in ethereum. Instead, miners vote on a parameter known as max gas, which is essentially a measure of the computation needed to verify a block. A higher max gas implies more transactions, and thus a larger block size.
This factor is constantly adjusted by ethereum miners based on how efficiently blocks are being propagated at the current size. If the network is too large for the current size, we would expect to see a lot of orphan blocks (or uncles, as ethereum calls them) since some miners on the network will still mine on an older block after a new one has already been found. If this is the case, we will see max gas reduce, and thus reduce the block size. If the opposite is true (where very few uncles are found), max gas may increase.
Eventually, the network will find an equilibrium between an acceptable orphan rate, and an optimal block size that allows to the maximum number of transactions in a block while still permitting propagation to take place efficiently.
You can see a chart of how this limit has changed over time over at etherscan.io's gaslimit chart. Essentially, as the network has developed better technologies to relay blocks quickly, we have seen the block size increase. However, it's not a very large increase, indicating that it is still bottlenecked by the network layer and block verification time.