Take the 2-minute tour ×
Bitcoin Stack Exchange is a question and answer site for Bitcoin crypto-currency enthusiasts. It's 100% free, no registration required.

First off, this really needs to get fixed. I'm downloading blocks for at least 8 hours now, and it's still not done. I know the problem of having to download the entire block chain is being worked on, but I feel it's already out of hand. Setting up an application should take something you would measure in minutes, not hours or days. (After spending the entire night, it's now 4:15am, I would estimate the setup time to be two days.)

Also downloading can be much optimized as it is. It's far from using my maximum download speed, and being connected to one peer is as fast as being connected to 8+ (yup, also tried port forwarding).

Now, ontopic.

Is it normal that downloading blocks goes slower, the further you progress? The first half (out of 194.000) went in the blink of an eye, after that it became less and less fast. Now at 20.000 blocks remaining, progress comes in bursts, but there can easily be 20 seconds of inactivity and when it bursts it's like 10 blocks.

Is this normal behavior? And also, why doesn't the number of connections make any difference for the speed at all? If it works like Bittorrent, which some describe it to work like, it should get faster with more people who can upload to you (more connections).

I've Googled, but all I've found is that it takes a few hours and that it may get stuck. It's not stuck and taking as long as people describe it to be, but I just wonder if it's normal that it's going slower and slower, and why it doesn't saturate my download capacity (like BitTorrent) no matter how many connections.

Thanks

share|improve this question
    
Are you running on an encrypted filesystem? –  Stephen Gornick Aug 17 '12 at 3:04
    
@StephenGornick Nope (already read that in another question posted here, but that's not it). –  Luc Aug 17 '12 at 3:06
    
Related question which provides an alternative way of downloading the blockchain: bitcoin.stackexchange.com/questions/799/… –  Highly Irregular Aug 17 '12 at 3:09
add comment

5 Answers

Very little of the time is spent downloading the blockchain data compared to how much time your computer is spending verifying each transaction. Bitcoin does a ton of disk i/o for this.

As there are more transactions since April (thanks to SatoshiDICE, we now see 40K and more transactions a day) those blocks take longer to verify.

If you are on an encrypted filesystem, that will really be hurtful for perfomance. If so, consider moving the blockchain and index to a filesystem that isn't encrypted and then symlink to them there.

Also, the latest release of the Bitcoin.org client (v0.6.3) has peformance improvements in this area over previous releases.

End uses generally will not need to run the client and instead use SPV (Simplified Payment Verification). So individuals and organizations that do want to run the full client should recognize that to do so requires a beefy CPU and quick storage (SSD maybe even).

A future version (after 0.7) will include LevelDB which should make huge gains as far as performance.

In the meantime, you can always securely download a full set of blockchain data from Sourceforge that is as-of a fairly recent point in time.

share|improve this answer
    
Oh, "If there is any attack that works in this way, it's a defect in the client". Didn't know that. I didn't try this yet because I didn't know the security implications. Only 4900 blocks left now, but let's see which method is faster. It's going ridiculously slow now... –  Luc Aug 17 '12 at 10:33
    
Just to update, version 0.7.1 which has been released, will automatically load bootstrap.dat, which is a blockchain data file pulled as a bittorrent data file. bitcointalk.org/index.php?topic=117982.0 –  Stephen Gornick Oct 24 '12 at 23:26
    
And ultraprune is implemented (though still some bugs being resolved) and first reports are that it performs phenomenally well, particularly where disk i/o was hugely limiting. –  Stephen Gornick Oct 24 '12 at 23:28
add comment
up vote 9 down vote accepted

Okay, combining answers and what more I've observed:

  • The rate of blocks slows down because more recent blocks are larger. The number of blocks remaining is not actually a good measure for estimated time.
  • You can make an estimate based on how much data is downloaded (in how much time) and how much data is left: At the moment of writing, my blk000x.dat files are totaling 8.3GB (10 Feb 2013).
  • I can't say anything about how fast the data size will increase, perhaps someone can comment on this for future readers.
    Update: It seems to be growing rather fast, probably linearly with the number of transactions per block
  • The reason it doesn't max out your download speed, is because it's actually doing a lot of other things to verify the transactions.
  • Downloading a zip with nightly blk000x.dat files is a lot faster, but copying the files didn't work... In the end I just left it running a few more hours. Also see the comment about a bug this might introduce when an error is present in the files (intentionally or not).
  • Encrypted filesystems make it take even longer because verifying blocks is an disk-intensive process.
  • If you can, use an SSD, this speeds it up by a lot. You could be done in a few hours instead of taking all night.
  • Usually this process just takes a while. 24 hours is not uncommon. Best is to leave your computer running during the night, and don't be disappointed when it's still not done in the morning.

Hope this helped! Credits to the other answers for some of the bullet points.

share|improve this answer
1  
If you download blk___.dat files, and put them in your Bitcoin folder, they are NOT verified. The client will not know that it didn't download+verify them itself. Doing this is a bad idea: if a small error is present (intentionally or not) in one of those files downloadable from a popular location. there is a risk that all those clients together will disagree about the validness of a particular chain, resulting in a large and permanent fork. There will soon be options for importing instead of replacing your block chain database, which will be safe. –  Pieter Wuille Aug 18 '12 at 21:15
1  
Most of the memory requirement comes from network send/receive buffers. Especially when many clients try to download the blockchain from you, memory usage will increase. If you limit the number of connections (-maxconnections=16) for example, memory usage will be more modest. –  Pieter Wuille Aug 18 '12 at 21:16
    
@PieterWuille Updated accordingly. Thanks for the comments :) –  Luc Aug 19 '12 at 22:53
add comment

Yes, it's normal for the rate of blocks being received to decrease as the download progresses.

The reason it slows down is because there were very few transactions in each block in the early days of Bitcoin, so the blocks were smaller. Later blocks are bigger.

The Satoshi Bitcoin Client is still a Beta version, and even if it wasn't it may still need to download the entire blockchain which I believe is over 2GB.

Which platform are you running the client on? You should be able estimate how much data is left to download by looking at the blockchain files on disk in the application folder.

It appears that HDD speed may be a significant factor too, as verification of each block is performed as it's downloaded, and that can be complicated for a transaction with many inputs. Discussion here. dooglus provides evidence of that theory by testing with a RAM disk and heavily reducing the download time.

share|improve this answer
    
I'm running on Linux, already checked the ~/.bitcoin folder. The block chain is in blk0001.dat right? It's 1.2GB now, so some time to go... Still wondering why it doesn't max out my download speed, and why it doesn't go faster with more connections. –  Luc Aug 17 '12 at 2:54
add comment
  1. Moving blkindex.dat to a let's say ram drive dramatically improves bitcoin/bitcoind performance. Just notice that currently index is around 1.5Gb size.
  2. Next version 0.8, as already been said, is using a new database instead of BDB and this works much faster and don't need to be moved to a ram drive or something.
share|improve this answer
    
How do I move blkindex.dat, and only blkindex.dat, to a ramdrive? Symlink/hardlink/whateverlink it there? –  Luc Jan 13 '13 at 23:20
1  
If your OS is Linux you can use tmpfs and create a symlink back to bitcoin directory. –  giner Jan 15 '13 at 3:04
add comment

I kinda figured out how to improve performance on ext4/linux:

http://tlfabian.blogspot.hu/2013/04/speeding-up-bitcoin-qt-on-linux.html

share|improve this answer
    
Although not a direct answer to the question, this is good to know. Thanks! –  Luc Apr 3 '13 at 14:43
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.