Okay, I will try to expand on my comment in an answer. Currently, it's difficult to determine why a payment is failing with our core lightning node because it doesn't provide much information.
However, in the last year, I worked on a lightning network metrics specification, which is available here or on Github. Using this specification, you can try to find the channel through which the payment is being sent and potentially gather useful metrics or ideas to improve the current system.
Regarding your question, there are several well-known problems that we're trying to work around or resolve.
You were correct in assuming that "local failed" means that your node could not forward the payment.
Regarding the general failure, this is not your fault, but one of the nodes along the path that fails;
This can be caused by several factors, such as:
- the sender selected a channel in the route path that isn't suitable for sharing the payment (note that the sender chooses the route and not the current node), and due to the unknown real capacity of the channel, the sender path is just a guess. Currently, this unknown real capacity of the channel is considered a privacy feature;
- the node that you have a channel with being offline (which can be common if the node uses a hidden network, aka tor),
- the sender sending a payment that has already dead from the source (also known as a probing attack, which aims to steal information from the network graph regarding which channel can forward the payment), or
- a bug in the software.
In some cases, like with a probing attack, you can try to mitigate the issue by using lnmetrics to look at the channel that has only been sending failed payments in the last day or 10 days. With lnmetrics, you can analyze the local failure and try to understand what went wrong with your node. Additionally, you can check the node uptime of the node that you have a channel with to get a better understanding of what the reason is.
However, this info does not provide any final answer, but sometimes removing a bad node can help. Ideally, this analysis with lnmetrics should be automated, but you can do it manually for now.
For "Failed transactions" I can find no reason. I guess these are transactions nodes behind my node can not route forward?
Mh, currently, I cannot find where we return this error message, you can add the correct error here that you find in your core lightning?
P.S: I noted that you pointed out several times the word transaction, as the HTLC is a pre-signed transaction, in lightning this is called payment, but maybe I am missing something here, so I hope you can clarify
