There would be a lot of empty blocks and the number of unconfirmed transactions would increase dramatically. But at the same time, miners who are including transactions in their blocks will start being paid more and more. Those miners including transactions in their blocks will be able to collect all of the transaction fees that are being paid. Because the number of transactions that can be confirmed in a given amount of time has significantly decreased, people will begin paying more and more in transaction fees in the hopes that their transaction gets selected. So the miners who are still including transactions are going to get paid even more than they were before.
This scenario is unlikely to happen, or if it did, it would not last long. Bitcoin's incentives structure makes it so that miners will want to include transactions in their blocks. Otherwise they will be missing out on money that they could be earning. We expect miners to be greedy (after all, most people want to have more money) and them being greedy is exactly what maintains the network.