1

I'm a developer and I try to install bitcoin-core 25.0 on ubuntu 22.04 on a mounted drive; as long as I run the command bitcoind -daemon as a root, everything is fine and I'm able to write some blocks in the mounted drive; however, when I create a user and even with all permission to the mounted drive, the bitcoind.service fails and I can't figure out the issue because the error is always (code=exited, status=1/FAILURE) with no other hint to help in that issue; if someone has created such a scenario, can you please assist?

Here's my bitcoin.conf file:

server=1
rpcauth=btc:c59370cd87d42948326c2f7d33fd8060$2e23117d4a676f9fb39949c91fb2e966a38c6302570f8d242d945bdd61a2f367
txindex=1
addresstype=bech32
changetype=bech32
zmqpubhashblock=tcp://127.0.0.1:29000
zmqpubrawtx=tcp://127.0.0.1:29000
rpcssl=0

#Directory where bitcoin store data
datadir=/mnt/blockstorage

# UTXO database cache size, in MiB
dbcache=2048
# Number of allowed pending RPC requests (default is 16)
rpcworkqueue=128

# How many seconds bitcoin will wait for a complete RPC HTTP request.
# after the HTTP connection is established.
rpcclienttimeout=30

Here's my bitcoind.service

[Unit]
Description=Bitcoin daemon
Documentation=https://github.com/bitcoin/bitcoin/blob/master/doc/init.md

# https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/local/bin/bitcoind -daemon \
                            -pid=/mnt/blockstorage/bitcoind.pid \
                            -conf=/root/.bitcoin/bitcoin.conf \
                            -datadir=/mnt/blockstorage \
                            -startupnotify='systemd-notify --ready' \
                            -shutdownnotify='systemd-notify --stopping'

# Make sure the config directory is readable by the service user
PermissionsStartOnly=true
#ExecStartPre=/bin/chgrp bitcoin /etc/bitcoin

# Process management
####################

Type=notify
NotifyAccess=all
PIDFile=/mnt/blockstorage/bitcoind.pid

Restart=on-failure
TimeoutStartSec=infinity
TimeoutStopSec=600

# Directory creation and permissions
####################################

# Run as bitcoin:bitcoin
User=btc
Group=btc

# /run/bitcoind
RuntimeDirectory=bitcoind
RuntimeDirectoryMode=0710

# /etc/bitcoin
ConfigurationDirectory=bitcoin
ConfigurationDirectoryMode=0710

# /var/lib/bitcoind
StateDirectory=bitcoind
StateDirectoryMode=0710

# Hardening measures
####################

# Provide a private /tmp and /var/tmp.
PrivateTmp=true

# Mount /usr, /boot/ and /etc read-only for the process.
ProtectSystem=full

# Deny access to /home, /root and /run/user
ProtectHome=false

# Disallow the process and all of its children to gain
# new privileges through execve().
NoNewPrivileges=true

# Use a new /dev namespace only populated with API pseudo devices
# such as /dev/null, /dev/zero and /dev/random.
PrivateDevices=true

# Deny the creation of writable and executable memory mappings.
MemoryDenyWriteExecute=true

[Install]
WantedBy=multi-user.target

Commands line to illustrate various folders and to view issue

root@node:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           794M  1.2M  793M   1% /run
/dev/vda1        71G   66G  1.1G  99% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/vdb1       1.6T  8.8G  1.5T   1% /mnt/blockstorage
tmpfs           794M  4.0K  794M   1% /run/user/0

root@node:~# ls /root/.bitcoin/
anchors.dat  banlist.json  bitcoin.conf  blocks  chainstate  debug.log  fee_estimates.dat  mempool.dat  peers.dat  settings.json  wallets

root@node:~# bitcoind -daemon
Bitcoin Core starting
root@node:~# ls /mnt/blockstorage
banlist.json  bitcoind.pid  blocks  chainstate  debug.log  fee_estimates.dat  indexes  lost+found  mempool.dat  peers.dat  settings.json

root@node:~# systemctl daemon-reload
root@node:~# systemctl enable bitcoind.service
root@node:~# systemctl start bitcoind.service
Job for bitcoind.service failed because the control process exited with error code.
See "systemctl status bitcoind.service" and "journalctl -xeu bitcoind.service" for details.
root@node:~# systemctl status bitcoind.service
× bitcoind.service - Bitcoin daemon
     Loaded: loaded (/etc/systemd/system/bitcoind.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2023-06-23 04:39:38 UTC; 16s ago
       Docs: https://github.com/bitcoin/bitcoin/blob/master/doc/init.md
    Process: 6205 ExecStart=/usr/local/bin/bitcoind -daemon -pid=/mnt/blockstorage/bitcoind.pid -conf=/root/.bitcoin/bitcoin.conf -datadir=/mnt/blockstorage -startupnotify=systemd-notify --r>
   Main PID: 6205 (code=exited, status=1/FAILURE)
        CPU: 112ms

Jun 23 04:39:38 node systemd[1]: Failed to start Bitcoin daemon.
Jun 23 04:39:38 node systemd[1]: bitcoind.service: Scheduled restart job, restart counter is at 5.
Jun 23 04:39:38 node systemd[1]: Stopped Bitcoin daemon.
Jun 23 04:39:38 node systemd[1]: bitcoind.service: Start request repeated too quickly.
Jun 23 04:39:38 node systemd[1]: bitcoind.service: Failed with result 'exit-code'.
Jun 23 04:39:38 node systemd[1]: Failed to start Bitcoin daemon.

btc user permissions on the mounted drive

root@node:/mnt/blockstorage# ls -l
total 23712
-rw------- 1 btc  btc        31 Jun 23 01:20 banlist.json
drwx------ 3 btc  btc     12288 Jun 23 04:44 blocks
drwx------ 2 btc  btc      4096 Jun 23 04:36 chainstate
-rw------- 1 btc  btc  23384304 Jun 23 04:44 debug.log
-rw------- 1 btc  btc    247985 Jun 23 02:16 fee_estimates.dat
drwx------ 3 btc  btc      4096 Jun 23 01:20 indexes
drwx------ 2 btc  btc     16384 Jun 22 21:45 lost+found
-rw------- 1 btc  btc        18 Jun 23 02:16 mempool.dat
-rw------- 1 btc  btc    591159 Jun 23 02:16 peers.dat
-rw------- 1 root root        4 Jun 23 04:36 settings.json

I've spent almost a week trying to figure this out because I don't understand why I can't get the bitcoind.service to start properly.

Please assist

---------------------- New Insight in the problem -----------------------

Though I haven't found the solution to my problem, at least I've found its root cause: it's the PID file.

What I did was to isolate all the issues from the bitcoind.service that I took from your github repository; and to do that, I create a minimalistic bitcoind.service file as illustrated below

[Unit]
Description=Bitcoin daemon
After=network.target
Wants=network-online.target

[Service]
User=btc
Group=btc
Type=forking
PIDFile=/mnt/blockstorage/bitcoin.pid
ExecStart=/usr/local/bin/bitcoind -daemon -pid=bitcoin.pid
KillMode=process
Restart=always
TimeoutSec=120
RestartSec=30

[Install]
WantedBy=multi-user.target

I ran the below commands as usual

root@node:~# systemctl daemon-reload
root@node:~# systemctl enable bitcoind.service
root@noode:~# systemctl start bitcoind.service
Job for bitcoind.service failed because a timeout was exceeded.
See "systemctl status bitcoind.service" and "journalctl -xeu bitcoind.service" for details.
root@node:~# systemctl status bitcoind.service
● bitcoind.service - Bitcoin daemon
     Loaded: loaded (/etc/systemd/system/bitcoind.service; enabled; vendor preset: enabled)
     Active: activating (start) since Sat 2023-06-24 00:11:35 UTC; 1min 29s ago
    Process: 14324 ExecStart=/usr/local/bin/bitcoind -daemon -pid=bitcoin.pid (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 9388)
     Memory: 28.0K
        CPU: 121ms
     CGroup: /system.slice/bitcoind.service

Jun 24 00:11:35 node systemd[1]: Starting Bitcoin daemon...
Jun 24 00:11:35 node bitcoind[14324]: Bitcoin Core starting
Jun 24 00:11:35 node systemd[1]: bitcoind.service: **Can't open PID file /mnt/blockstorage/bitcoin.pid (yet?) after start: Operation not permitted**

Here's what happens:

a. As long as the I run bitcoin-core manually through the root user, the PID file is created and the bitcoind -daemon runs without any issue

b. However, when I mount a drive and assign it to another user, the PID file can no longer be accessed since it's created by the root user and consequently, the assigned user can't open it;

The issue is that the PID file is created by the daemon; so we can't even run a chown on that file to assign it to the mount drive's user. So that's why the bitcoind.service can't run on the the mount drive but does run manually when called by the root user.

Is this a bug?

-------------------- Change permissions as instructed by @deyw -------------

root@node:~# chown btc:btc /mnt/blockstorage
root@node:~# ls -l /mnt/blockstorage
total 144764
-rw------- 1 root root        85 Jun 26 04:17 anchors.dat
drwx------ 2 root root      4096 Jun 26 03:56 node-wallet
-rw------- 1 btc  btc         31 Jun 23 01:20 banlist.json
drwx------ 3 btc  btc     229376 Jun 26 01:15 blocks
drwx------ 2 btc  btc     118784 Jun 26 04:14 chainstate
-rw------- 1 btc  btc     115762 Jun 26 04:17 debug.log
-rw------- 1 btc  btc   10104274 Jun 25 00:00 debug.log.1
-rw------- 1 btc  btc     247985 Jun 26 04:17 fee_estimates.dat
drwx------ 3 btc  btc       4096 Jun 23 01:20 indexes
drwx------ 2 btc  btc      16384 Jun 22 21:45 lost+found
-rw------- 1 root root 135606693 Jun 26 04:17 mempool.dat
-rw------- 1 root root   1761875 Jun 26 04:17 peers.dat
-rw------- 1 root root         4 Jun 26 03:58 settings.json
root@node:~# su - btc
btc@node:~$ chmod 0750 /mnt/blockstorage
btc@node:~$ exit
logout
root@node:~# ls -l /mnt/blockstorage
total 144764
-rw------- 1 root root        85 Jun 26 04:17 anchors.dat
drwx------ 2 root root      4096 Jun 26 03:56 node-wallet
-rw------- 1 btc  btc         31 Jun 23 01:20 banlist.json
drwx------ 3 btc  btc     229376 Jun 26 01:15 blocks
drwx------ 2 btc  btc     118784 Jun 26 04:14 chainstate
-rw------- 1 btc  btc     115762 Jun 26 04:17 debug.log
-rw------- 1 btc  btc   10104274 Jun 25 00:00 debug.log.1
-rw------- 1 btc  btc     247985 Jun 26 04:17 fee_estimates.dat
drwx------ 3 btc  btc       4096 Jun 23 01:20 indexes
drwx------ 2 btc  btc      16384 Jun 22 21:45 lost+found
-rw------- 1 root root 135606693 Jun 26 04:17 mempool.dat
-rw------- 1 root root   1761875 Jun 26 04:17 peers.dat
-rw------- 1 root root         4 Jun 26 03:58 settings.json
root@node:~# root@node:~# systemctl daemon-reload
root@node:~# systemctl enable bitcoind.service
root@node:~# systemctl start bitcoind.service
Job for bitcoind.service failed because the control process exited with error code.
See "systemctl status bitcoind.service" and "journalctl -xeu bitcoind.service" for details.
root@node:~# systemctl status bitcoind.service
● bitcoind.service - Bitcoin daemon
     Loaded: loaded (/etc/systemd/system/bitcoind.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Mon 2023-06-26 04:26:26 UTC; 16s ago
    Process: 40764 ExecStart=/usr/local/bin/bitcoind -daemon -pid=/mnt/blockstorage/bitcoind.pid (code=exited, status=1/FAILURE)
        CPU: 105ms
4
  • Your settings.json file is not readable by user btc. Could that be it? Commented Jun 23, 2023 at 11:41
  • this is not the issue. I changed the ownership of settings.json but this doesn't fix the problem Commented Jun 23, 2023 at 23:22
  • it may help to give the btc user the permissions on the /mnt/blockstorage directory: chown btc:btc /mnt/blockstorage & chmod 0750 /mnt/blockstorage
    – deyw
    Commented Jun 25, 2023 at 15:48
  • It didn't work; please see all the details in above Commented Jun 26, 2023 at 4:28

1 Answer 1

2

After a lot of research, I was finally able to make this works. Here were the issues:

  • The bitcoind program was installed by the root user; therefore, /usr/local/bin/bitcoind is a file owned by the root and not the bitcoin user. I therefore changed the User and Group, see below, to point to root.

  • b. Additionally, bitcoind was put in the system path creating therefore an error when "/usr/local/bin" is put in front of the "bitcoind" command.

The working bitcoind.service looks like the file below:

[Unit]
Description=Bitcoin daemon
After=network-online.target
Wants=network-online.target

[Service]
User=root
Group=root
Type=forking

ExecStart=bitcoind -daemon 
                        -pid=/mnt/blockstorage/bitcoind.pid \
                        -conf=/root/.bitcoin/bitcoin.conf \
                        -datadir=/mnt/blockstorage \
                        -startupnotify='systemd-notify --ready' \
                        -shutdownnotify='systemd-notify --stopping'

Restart=on-failure
TimeoutStartSec=infinity
TimeoutStopSec=600

[Install]
WantedBy=multi-user.target

I hope this helps someone out there because I really struggle with this one; and the bitcoin sample bitcoind.service on github was more confusing that helpful.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.