0

I am running bitcoind (Bitcoin Core) on a Raspberry 3B and the process occationally "disappears" from the list in top. The log debug.log shows no etries, last entry is usually something like

2019-11-08T20:21:33Z UpdateTip: new best=0000000000000000001dd32ef554a1bb9e204d6f46ab345302019412ac2f6af9 height=506280 version=0x20000000 log2_work=87.972007 tx=295647959 date='2018-01-27T01:16:11Z' progress=0.631698 cache=13.3MiB(59241txo)

What could be a reason and where could I search?

My config for my low-memory-system is:

 $ cat bitcoin.conf
daemon=1
#printtoconsole=1

# is required by Fail2Ban described below
logips=1
# magic RBP optimisations
maxconnections=40
maxuploadtarget=5000

# running without swap
dbcache=100
maxorphantx=10
maxmempool=50

txindex=1

rpcuser=...
rpcpassword=...
rpcclienttimeout=300
server=1
#datadir=/home/bitcoin

Maybe I have a problem with my external HDD:

/var/log/kern.log shows

Nov 17 21:23:22 RPi3 kernel: [642589.539592] EXT4-fs error (device sdb1): ext4_find_entry:1455: inode #2: comm smbd: reading directory lblock 0
Nov 17 21:23:22 RPi3 kernel: [642589.539653] EXT4-fs error (device sdb1): ext4_find_entry:1455: inode #2: comm smbd: reading directory lblock 0
Nov 17 21:23:22 RPi3 kernel: [642589.539798] EXT4-fs warning (device sdb1): htree_dirblock_to_tree:995: inode #2: lblock 0: comm smbd: error -5 reading directory block
Nov 17 21:23:22 RPi3 kernel: [642589.539895] EXT4-fs error (device sdb1): __ext4_get_inode_loc:4674: inode #2: block 2081: comm smbd: unable to read itable block
Nov 17 21:23:22 RPi3 kernel: [642589.539913] EXT4-fs error (device sdb1) in ext4_reserve_inode_write:5883: IO failure
Nov 17 21:23:22 RPi3 kernel: [642589.539932] ------------[ cut here ]------------
Nov 17 21:23:22 RPi3 kernel: [642589.539953] WARNING: CPU: 0 PID: 29585 at fs/fs-writeback.c:2238 __mark_inode_dirty+0x2f4/0x408
Nov 17 21:23:22 RPi3 kernel: [642589.539959] bdi-block not registered
Nov 17 21:23:22 RPi3 kernel: [642589.539967] Modules linked in: fuse rfcomm bnep hci_uart btbcm serdev bluetooth ecdh_generic 8021q garp stp llc sg brcmfmac brcmutil sha256_generic cfg80211 $
Nov 17 21:23:22 RPi3 kernel: [642589.540331]  ip_tables x_tables ipv6
Nov 17 21:23:22 RPi3 kernel: [642589.540359] CPU: 0 PID: 29585 Comm: smbd Tainted: G         C        4.19.66-v7+ #1253
Nov 17 21:23:22 RPi3 kernel: [642589.540364] Hardware name: BCM2835
Nov 17 21:23:22 RPi3 kernel: [642589.540391] [<80111f38>] (unwind_backtrace) from [<8010d4b0>] (show_stack+0x20/0x24)
Nov 17 21:23:22 RPi3 kernel: [642589.540410] [<8010d4b0>] (show_stack) from [<808191e0>] (dump_stack+0xd4/0x118)
Nov 17 21:23:22 RPi3 kernel: [642589.540428] [<808191e0>] (dump_stack) from [<801209c8>] (__warn+0x104/0x11c)
Nov 17 21:23:22 RPi3 kernel: [642589.540444] [<801209c8>] (__warn) from [<80120a38>] (warn_slowpath_fmt+0x58/0x74)
Nov 17 21:23:22 RPi3 kernel: [642589.540460] [<80120a38>] (warn_slowpath_fmt) from [<802dd4f8>] (__mark_inode_dirty+0x2f4/0x408)
Nov 17 21:23:22 RPi3 kernel: [642589.540477] [<802dd4f8>] (__mark_inode_dirty) from [<802c8434>] (generic_update_time+0x8c/0x100)
Nov 17 21:23:22 RPi3 kernel: [642589.540493] [<802c8434>] (generic_update_time) from [<802ca614>] (touch_atime+0xc8/0xe4)
Nov 17 21:23:22 RPi3 kernel: [642589.540510] [<802ca614>] (touch_atime) from [<802c0138>] (iterate_dir+0xf8/0x160)
Nov 17 21:23:22 RPi3 kernel: [642589.540528] [<802c0138>] (iterate_dir) from [<802c0728>] (ksys_getdents64+0x98/0x140)
Nov 17 21:23:22 RPi3 kernel: [642589.540544] [<802c0728>] (ksys_getdents64) from [<802c07e8>] (sys_getdents64+0x18/0x1c)
Nov 17 21:23:22 RPi3 kernel: [642589.540560] [<802c07e8>] (sys_getdents64) from [<80101000>] (ret_fast_syscall+0x0/0x28)
Nov 17 21:23:22 RPi3 kernel: [642589.540568] Exception stack(0x81adffa8 to 0x81adfff0)
Nov 17 21:23:22 RPi3 kernel: [642589.540579] ffa0:                   017bc2b8 017bc2d8 00000021 017bc2d8 00008000 00000000
Nov 17 21:23:22 RPi3 kernel: [642589.540592] ffc0: 017bc2b8 017bc2d8 017bc2bc 000000d9 017bc2b8 7eae92e0 017b5db0 76d6c534
Nov 17 21:23:22 RPi3 kernel: [642589.540601] ffe0: 76dfaf58 7eae91b0 76237474 762373a8
Nov 17 21:23:22 RPi3 kernel: [642589.540609] ---[ end trace 0aca54ceb067b5f8 ]---
Nov 17 21:23:22 RPi3 kernel: [642589.541693] EXT4-fs error (device sdb1): ext4_find_entry:1455: inode #2: comm smbd: reading directory lblock 0
Nov 17 21:23:22 RPi3 kernel: [642589.541744] EXT4-fs error (device sdb1): ext4_find_entry:1455: inode #2: comm smbd: reading directory lblock 0
Nov 17 21:23:22 RPi3 kernel: [642589.541881] EXT4-fs warning (device sdb1): htree_dirblock_to_tree:995: inode #2: lblock 0: comm smbd: error -5 reading directory block
Nov 17 21:23:27 RPi3 kernel: [642594.368519] [UFW BLOCK] IN=eth0 OUT= MAC=b8:27:eb:3d:f7:ef:00:04:20:1e:dd:54:08:00:45:00:00:28:31:d9:00:00:40:06:fe:7c SRC=192.168.100.128 DST=192.168.100.16$
Nov 17 21:23:27 RPi3 kernel: [642595.072268] jbd2_journal_bmap: journal block not found at offset 13813 on sdb1-8
Nov 17 21:23:27 RPi3 kernel: [642595.072290] Aborting journal on device sdb1-8.

also there is an issue with a timeout waiting for hardware interrupt but I don't know anything about it, except there seems to be a related Github issue:

Nov 18 09:51:04 RPi3 kernel: [687451.651659] mmc0: timeout waiting for hardware interrupt.
...
Nov 18 10:22:42 RPi3 kernel: [689349.901550] [c03fe701] PRD3 b5880284 0
Nov 18 10:22:42 RPi3 kernel: [689349.901557] [c03fe702] PDM> b9738fa0 0
Nov 18 10:22:42 RPi3 kernel: [689349.901565] [c03fe703] REQ< b9738ec0 10801
Nov 18 10:22:42 RPi3 kernel: [689349.901572] [c03fe704] CMD< 17 8
Nov 18 10:22:42 RPi3 kernel: [689349.901580] [c03fe708] FCM< b9738ec0 b9738f04
Nov 18 10:22:42 RPi3 kernel: [689349.901587] [c03fe70b] RSP  900 0
Nov 18 10:22:42 RPi3 kernel: [689349.901595] [c03fe70b] CMD< 12 929e08 
Nov 18 10:22:42 RPi3 kernel: [689349.901603] [c03fe70d] CMDD 8 200
Nov 18 10:22:42 RPi3 kernel: [689349.901610] [c03fe70e] SDMA b9738fa0 b9ebe840
Nov 18 10:22:42 RPi3 kernel: [689349.901618] [c03fe710] FCM< b9738ec0 b9738f38
Nov 18 10:22:42 RPi3 kernel: [689349.901625] [c03fe710] RSP  900 0
Nov 18 10:22:42 RPi3 kernel: [689349.901632] [c03fe711] FCM> b9738ec0 0
Nov 18 10:22:42 RPi3 kernel: [689349.901640] [c03fe711] FCM> b9738ec0 0
Nov 18 10:22:42 RPi3 kernel: [689349.901647] [c03fe711] CMD  12 200
Nov 18 10:22:42 RPi3 kernel: [689349.901655] [c03fe712] REQ> b9738ec0 0
Nov 18 10:22:42 RPi3 kernel: [689349.901662] [c0db9160] TIM< 0 0
Nov 18 10:22:42 RPi3 kernel: [689349.901676] mmc0: sbc op 23 arg 0x8 flags 0x15 - resp 00000900 00000000 00000000 00000000, err 0
Nov 18 10:22:42 RPi3 kernel: [689349.901688] mmc0: cmd op 18 arg 0x929e08 flags 0xb5 - resp 00000900 00000000 00000000 00000000, err 0
Nov 18 10:22:42 RPi3 kernel: [689349.901695] mmc0: data blocks 8 blksz 200 - err 0
Nov 18 10:22:42 RPi3 kernel: [689349.901706] mmc0: stop op 12 arg 0x0 flags 0x95 - resp 00000000 00000000 00000000 00000000, err 0
Nov 18 10:22:42 RPi3 kernel: [689349.901711] mmc0: =========== REGISTER DUMP ===========
Nov 18 10:22:42 RPi3 kernel: [689349.901716] mmc0: SDCMD  0x00004052   
Nov 18 10:22:42 RPi3 kernel: [689349.901722] mmc0: SDARG  0x00929e08   
Nov 18 10:22:42 RPi3 kernel: [689349.901727] mmc0: SDTOUT 0x017d7840
Nov 18 10:22:42 RPi3 kernel: [689349.901732] mmc0: SDCDIV 0x00000003   
Nov 18 10:22:42 RPi3 kernel: [689349.901737] mmc0: SDRSP0 0x00000900   
Nov 18 10:22:42 RPi3 kernel: [689349.901742] mmc0: SDRSP1 0x00001269   
Nov 18 10:22:42 RPi3 kernel: [689349.901747] mmc0: SDRSP2 0x7fffffff   
Nov 18 10:22:42 RPi3 kernel: [689349.901753] mmc0: SDRSP3 0x00024007   
Nov 18 10:22:42 RPi3 kernel: [689349.901758] mmc0: SDHSTS 0x00000080
Nov 18 10:22:42 RPi3 kernel: [689349.901763] mmc0: SDVDD  0x00000001
Nov 18 10:22:42 RPi3 kernel: [689349.901768] mmc0: SDEDM  0x00010801   
Nov 18 10:22:42 RPi3 kernel: [689349.901773] mmc0: SDHCFG 0x0000040e   
Nov 18 10:22:42 RPi3 kernel: [689349.901778] mmc0: SDHBCT 0x00000200
Nov 18 10:22:42 RPi3 kernel: [689349.901783] mmc0: SDHBLC 0x00000008   
Nov 18 10:22:42 RPi3 kernel: [689349.901788] mmc0: ===========================================
2

That sounds like it's going out of memory and is killed by the system. Especially on a low-memory device like an RPi3 this seems very likely.

For information on how to reduce bitcoind memory usage, see https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-memory.md

  • I will check it. Especially the last comment regarding the Linux-specific thread thing seems interesting. I also added my config to my question since this already was a setup for a low-memory environment. – Ben Nov 9 '19 at 18:56
  • Read your kernel log. – Anonymous Nov 10 '19 at 6:18
  • @Anonymous You mean with journalctl -k, right? Did not find anything relevant around the time the last entry was in debug.log from bitcoind. – Ben Nov 12 '19 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.