Why is my "spendable_msat" much lower than "msatoshi_to_us"?

Question

Today I was attempting to send a Lightning Network payment via c-lightning, version v0.7.1-64-gbb30104, using a channel I established in 2018. Strangely, my payment kept failing even if the destination was routable. Finally, I noticed the reason was a relatively large disparity between spendable_msat and msatoshi_to_us.

"msatoshi_to_us" : 49947931,
"to_us_msat" : "49947930msat",
"msatoshi_to_us_min" : 49947930,
"min_to_us_msat" : "49947930msat",
"msatoshi_to_us_max" : 50000000,
"max_to_us_msat" : "50000000msat",
"msatoshi_total" : 50000000,
"total_msat" : "50000000msat",
"dust_limit_satoshis" : 546,
"dust_limit_msat" : "546000msat",
"max_htlc_value_in_flight_msat" : 18446744073709551615,
"max_total_htlc_in_msat" : "18446744073709551615msat",
"their_channel_reserve_satoshis" : 546,
"their_reserve_msat" : "546000msat",
"our_channel_reserve_satoshis" : 546,
"our_reserve_msat" : "546000msat",
"spendable_msatoshi" : 35617930,
"spendable_msat" : "35617930msat",
"htlc_minimum_msat" : 0,
"minimum_htlc_in_msat" : "0msat",
"their_to_self_delay" : 144,
"our_to_self_delay" : 144,
"max_accepted_htlcs" : 483,
"status" : [
  "CHANNELD_NORMAL:Reconnected, and reestablished.",
  "CHANNELD_NORMAL:Funding transaction locked. Channel announced."
],

As you see, my "spendable_msatoshi" is much lower than "msatoshi_to_us", a difference of 14373070 msatoshis. Even considering "their_reserve_msat" and "our_reserve_msat", it can only explain an amount of 1092000 msatoshis, or ~7% of the unspendable fund, where does the rest go?

Why does this even occur? And what should I do to fix it?

I tried reading the source code of c-lightning, and clearly HTLC and transaction fee is also deduced from "spendable_msat", however, those values are invisible in lightning-cli listpeers and I still cannot determine the reason.

/* Compute how much we can send via this channel. */
if (!amount_msat_sub_sat(&spendable,
             channel->our_msat,
             channel->channel_info.their_config.channel_reserve))
    spendable = AMOUNT_MSAT(0);

/* Take away any currently-offered HTLCs. */
subtract_offered_htlcs(channel, &spendable);

/* If we're funder, subtract txfees we'll need to spend this */
if (channel->funder == LOCAL) {
    if (!amount_msat_sub_sat(&spendable, spendable,
                 commit_txfee(channel, spendable)))
        spendable = AMOUNT_MSAT(0);
}

/* We can't offer an HTLC less than the other side will accept. */
if (amount_msat_less(spendable,
             channel->channel_info.their_config.htlc_minimum))
    spendable = AMOUNT_MSAT(0);

/* We can't offer an HTLC over the max payment threshold either. */
if (amount_msat_greater(spendable, get_chainparams(ld)->max_payment))
    spendable = get_chainparams(ld)->max_payment;

json_add_amount_msat_compat(response, spendable,
                "spendable_msatoshi", "spendable_msat");
json_add_amount_msat_compat(response,
                channel->our_config.htlc_minimum,
                "htlc_minimum_msat",
                "minimum_htlc_in_msat");

Answer 1

You basically answered the question already yourself. Besides the 1% channel reserve you as the funder are responsible for paying onchain fees in case you need to force close the channel by spending the commitment transaction.

C Lightning (and lightning nodes in general) are rather overestimating onchain fees as they cannot rely on the fact that the fees that where taken when the commitment transaction was signed will work in the future if force closing of the channel takes place.

Look for example at this transaction in which my c Lightning node had a channel to test the lightning app (running lnd) and a cross implementation bug produced a forced channel close. You can see that I paid about 100 satoshis per byte in fees resulting in 18312 satoshis. This is the same ballpark number as the value which you describe here. (looking at the mempool this number sounds realistically to me)

Things you can do

1. Don't force close channels (fees will be much smaller with mutual closes, since the fee is more accurate and only one tx is needed)
2. Have bigger channels then the relative fee is not so high
3. Look at the code where fees are set. Propose better strategies in the c Lightning issue tracker or provide an api to set / read them. (as far as I remember there is currently no api for that)
4. You cannot even do too much as the fees being used are negotiated between you and your channel partner. If the channel partner starts out with a really high fee there is hardly anything you can do to defend against this.

Btw welcome to the rabbit hole of onchain fees with lightning. Doing this for a while now I realized it is one of the most subtle and present challenges that we are facing.