1

GitHub suggested discussing here before bothering developers with an issue. So be it! :)

I'm running Bitcoin Core Version v0.20.1 and Linux Mint 18.3 on crappy old hardware. Pruning blockchain to 2 G and setting dbcache=2048 didn't help much. Moving datadir from HDD onto a USB-Stick did a little.

top says CPU around 20% can reach up to 40 if I manually drop slow or unresponsive nodes. CPU frequency is rarely going up. So, no issue here.

Memory usage is fine too, actually a lot of headroom. Looks like bitcoin-qt doesn't need that much cache.

Swap is empty. (vm.swappiness=5)

iostat

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdc               0,00     0,00  549,00    0,00 51125,00     0,00   186,25     1,06    1,92    1,92    0,00   1,70  93,60

Bottleneck seems to be file I/O. Which confuses me since that is what a cache is meant to fix, isn't it?

Now to my questions:

Is there anything I am missing to improve disk access? Is there an open issue on GitHub regarding this already?

And secondly for the experts on dropping lazy nodes. Can you give me a hint at which classes I have to look to understand the strategy? Or even better a link to the discussions on that topic.
For I don't want to start an arms race between clients seeking faster download and nodes trying to serve fair.

Short summary what I've learned so far.

  • Don't use USB-Sticks for datadir. For bitcoin-qt might shutdown due to Fatal LevelDB errors.
  • Pruning increases disk I/O while saving disk space.
  • Even on crappy old hardware memory and CPU are most probably not the limiting factor.
Improve this question
7
  • 2
    Pruning prevents aggressive caching, as it requires a flush any time files are deleted from disk. Have you tried without pruning? – Pieter Wuille Oct 30 '20 at 23:35
  • Thanks Pieter! My USB-Stick is too small for the hole blockchain. That is why I went for pruning. But it looks like I don't need to store the blockchain itself on the stick, true? Do you have any hints which files are impacting heavy disk I/O? – pisjatblin Oct 30 '20 at 23:42
  • 1
    I would recommend against using a USB stick for anything; they're notoriously unreliable for heavy database workloads like Bitcoin Core needs. With sufficient dbcache the speed of the disk really shouldn't matter. The heavy I/O is in the chainstate/ directory only. – Pieter Wuille Oct 30 '20 at 23:58
  • I totally agree on not using USB-Sticks. :) But my crappy hardware somewhat forces me to. As I said disk I/O is the issue for me. The stick is performing better, now we are talking about days not weeks. So technically the correct answer would be: "Dummy, buy better hardware!". But I'm more concerned about the underlying bottleneck. And crappy HW is what you need to find bottlenecks. :) – pisjatblin Oct 31 '20 at 0:04
  • 1
    Well have you tried using your internal hard drive without pruning, but with a large dbcache? Even if you don't have enough space for the entire chain, it may give you an idea of the speed. – Pieter Wuille Oct 31 '20 at 0:09
0

Short and simple answer to my initial problem regarding downloading the blockchain: You might need better hardware! :)

In summary bitcoin-qt runs out of the box. So do not fumble around. Unless you really need to.

Pruning saves disk space but increases disk I/O. Increasing dbcache can help but not so much for pruning. Both can be done in config-window. Thus, no need to change bitcoin.conf.

Plus, ... I haven't mentioned it lately, have I? ... don't use USB-Sticks! :)

Having that out of my way let's dive into my "real" question.

If we look at it from a queueing theory point of view we have three fields to consider.

  1. Input that are the nodes you are feasting on.
  2. Processing where your hardware and configuration comes into play.
  3. Output writing to disk in our case.

Input side At first glance it look like I had problems connecting to responsive nodes. Looking deeper into it, I found the sending side was never really an issue.

The sending behaviour of nodes in average over a long period made me distinguish three main types. A few are sending MB, some a few KB and a lot just 150 Bytes then drop out

Over time responsive nodes lower their data rate.

Data is usually coming in bulks. All nodes send in parallel many MB/s and then stop for several minutes. While my CPU and disk are constantly busy. So it looks like they are filling one input queue. This is generally a usual behaviour for queued systems. They are pumping. That's why you have buffers. Looks like increasing buffers will not help on my machine, since there is already plenty of headroom.

Sending at high data rates and slowing down over time makes sense for the nodes to distribute load on other nodes. From a clients view this is a preferable behaviour too. Although my view as a user is bitcoin-qt can improve its dropping strategy, to not bother nodes who did their fair share and focus more on still very responsive nodes.

Sending KB even somewhat erratically at low data rates isn't really helpful. Since in a free network you can't tell a node what to do, clients need a strategy to drop those nodes early.

Why do some nodes just show up to say hello and leave 150 behind? Probably a kind of handshake. But do we actually need to hold them for a while?

In short: Yes, technically there is room for improvement. But is it worth the effort?

Processing side I'd say everything is fine. Neither memory nor CPU are problematic.

Output side Disk I/O is an issue, at least for me. As Pieter pointed out pruning prevents optimal caching. I'm reluctant to judge on this topic without thorough understanding. But my first approach would be reducing the number of files involved.

Many thanks to Pieter and Murch for their quick response. Helped a lot!

Feedback and corrections highly appreciated!

Improve this answer
1
  • An addition to USB-Sticks. Some db errors only require restarting bitcoin-qt. But you can also end up reindexing a pruned database. You don't want that to happen. – pisjatblin Nov 2 '20 at 7:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.