Speaking as someone who has worked on a blockchain project for a bank, I can tell you that banks are interested for several reasons.
Banks know they are ripe for disruption
Due to a combination of heavy regulation and incumbency (mixed with a variable level of collusion), banks have remained relatively protected from the technological disruption that has been occurring in every other industry. This can't last, and banks know it. They are trying to get out in front of whatever technology might disrupt them, and they have enough money to throw around while investigating. As a result, banks are spending a lot of time and money exploring the possibilities of blockchain technologies.
Banks want to reduce their transaction fees
Customers of banks are not the only ones who have to pay transaction fees. Banks, themselves, are charged for moving money around, and are always looking for a way to reduce their costs. If they can get together and reduce their fees by creating something like the R3 Consortium, then they will do so. Currently, these banks would all have to transfer money to each other over ACH, Fedwire, SWIFT, or some other legacy network. This incurs fees and counter-party risk. If the general public can use a blockchain to remove counter-parties, why shouldn't banks?
Blockchain solutions offer faster transactions than the current infrastructure allows
The project I worked on for a bank was implementing a private blockchain for supply chain financing. Sending money internationally to pay for manufactured goods and parts has always been a hassle, even for the banks. There is a lots of risk involved, and delivery of product is often held up by archaic money transfers. If banks can reduce this time and risk by way of a private blockchain, then factories and suppliers can get paid sooner, and goods can be delivered in a more timely fashion. This is very appealing to the customers of a bank who are doing business around the world, and even to the bank itself. Suppliers offer discounts for faster payments, called dynamic discounting. By using a faster payment solution, banks can lend credit to the purchaser, obtain the goods for the discount, and split the difference with their customer. The supplier gets paid sooner, the customer gets their goods for cheaper, and the bank gets to charge a larger fee than the otherwise would have. It's a 3-way win.
Blockchains offer a better solution to traditional bookkeeping and auditing
A bank can be thought of as a big ledger of whom owes how much money to whom. When money is transferred between banks and other financial institutions, what is really happening is that both institutions update both of their ledgers to show that recipient owes a little more, or the sender owes a little less. These are two separate ledgers, and banks have entire departments that keep track of them and audit them, making sure they are in sync. That costs money and takes time. A blockchain can also be thought of as a big ledger. Banks are curious if blockchains can replace the large amount of work that has always been done to keep these records updated and correct.
Blockchains offer a reduction in fraud
Probably the number one thing banks have been worried about in recent years is fraud. Fraud has been the fastest growing crime since the turn of the millennium, and banks are often left picking up the tab. When Target and Home Depot had their point of sale systems compromised, and millions of customers had their credit and debit cards stolen, it was the banks that had to pay the brunt of it. They not only had to incur the costs of the fraud, itself (along with the merchants), but also the costs to re-issue cards to millions of customers. Banks are interested in any technology that promises to reduce fraud, and blockchains get suggested often. Just how the banks plan to use blockchains to reduce fraud, I don't know, but I can assure you that they are going to try to find a way.
Conclusions
My personal opinion is that private blockchains offer a lot less than a public blockchain such as Bitcoin does. However, they still have many use cases within the world of traditional banking. Because the technology, and Bitcoin, have the potential to be very disruptive, banks are becoming acquainted with it as an act of preparation for the future. Bitcoin suffers from an image problem, as well as having many unanswered questions in the realm of traditional financial regulation. Private blockchains offer banks the opportunity to "dip their toes in the water" without the risk of running afoul of regulators.
