17

Has anyone stored the historical market depth & tick data from MtGox? Is it available publically? Would be useful if you were making a bot.

  • Related, there is a BitcoinCharts API which provides some timeseries information bitcoincharts.com/about/markets-api – ACyclic Oct 19 '11 at 0:30
  • I have emailed MtGox and TradeHill about this, have not received a reply. Clark Moody doesn't store historic market depth information as there's too much data. – ACyclic Oct 26 '11 at 3:00
  • I think that's the main problem. It really is too much data. – nmat Oct 26 '11 at 14:22
  • 1
    I grabbed a simple script I had and I got it fetching market depth every 5 seconds and saving the differences (I should use websockets, but I didn't have a script ready). After 2 days I have ~5MB of data so maybe this is not that hard to do. The data could be even saved in a more compressed format (I am just dumping what MtGox gives me). It's just a matter of setting up a server running 24/7 with a script like this. – nmat Oct 28 '11 at 14:49
12
+100

Current market depth available at

All trades which happened in the last 24 hours available at

Retrieves 100 trades that happend after specified transaction

A script is required to retrive all historical data, like the one posted by mizerydearia. The idea is to cycle download request while incrementing "transaction tid" aka "lasttrade" to download all data in steps.

#!/bin/bash
outputfile=data
lasttrade=0
while test -n "$lasttrade";do
    wget --quiet https://mtgox.com/code/data/getTrades.php?since=$lasttrade
    if test "`cat getTrades.php\?since\=$lasttrade`" != "[]";then
        cat getTrades.php\?since\=* >> $outputfile;
        echo >> $outputfile;
        lasttrade=`cat getTrades.php\?since\=$lasttrade|sed "s/,/\n/g"|grep tid|tail -n 1|cut -d ":" -f 2|cut -d "\"" -f 2`;
        lasttradets=`cat getTrades.php\?since\=*|sed "s/,/\n/g"|grep date|tail -n 1|cut -d ":" -f 2`;
        echo "$(date -d @$lasttradets)  tid $lasttrade"
    else
        echo Finished
    fi
    rm getTrades.php\?since\=*;
done

See MtGox - API for more http API calls.

  • Any idea how big this file is today? I'm at about 3.5 gb right now and still going. – jordan.baucke Feb 22 '13 at 16:01
  • @jordan.baucke what size did you get? – kirill_igum Dec 8 '13 at 3:33
  • I'm running a similar script -- it's currently up to 10/20/2011 @ ~ 155 MB uncompressed -- I'll let you know when it completes, and see if I can dump it up online somewhere. – BrainSlugs83 Jan 13 '14 at 4:32
  • Well, I got to 3/11/2013 -- and it's only about ~670 MB so far. So there must be a bug in the above script. Probably related to the fact that it's parsing the JSON with Regex. Maybe an error is being thrown every so often, and it's starting over at the error code (using it as a TID) or something? – BrainSlugs83 Jan 13 '14 at 9:46
  • I got up to date (1/14/2014) this morning -- completely uncompressed, the JSON is about 1.33 GB -- compressed in a regular .zip file, it's about 160 MB. I used the URL and the approach described above (slightly different implementation), and although it does not mention USD in the URL, the results appear to be in USD only (maybe that happened when they changed the API? This could also be the reason for the much smaller file size). At any rate, I've posted my dump here: bit.ly/1dMzqhJ – BrainSlugs83 Jan 14 '14 at 12:15
3

To get ALL mtgoxUSD trades since the start you can use the following url:

www.bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD&start=0

WARNING. This generates a 49mb csv at the moment and can take some time, so please only follow the link if you actually need the data.

Visit http://bitcoincharts.com/about/markets-api/ for more information.

  • i think this is daily not tick data – kirill_igum Dec 8 '13 at 3:33
2

Just noting here that MtGox tid's are not included in the data you get from bitcoincharts in that csv call API, and they switched from numerical indexing to microsecond timestamping a while back. In trying to maintain a db with integrity, you need the unique tid data... The history file is now over 100MB also.

1

Original MtGox Shared Dataset

Once upon a time there was a Google BigQuery dataset that was updated often with MtGox's official trade record. That's no longer being published because... ask someone who should explain it.

Data on Google BigQuery

The BigQuery shared MtGox Trades dataset is (@2014-04-20) available at https://bigquery.cloud.google.com/dataset/931531665892:mtgox

The BitTorrent Version

If you'd rather not BigQuery, but you still want that data, there's a torrent, here's the full-metal-magnet.

magnet:?xt=urn:btih:c87cd62f9484993f0153a47fc51f06fb43ed71ee&dn=931531665892-mtgox-trades.csv&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A80

The MtGox Leak Database

In 2014 the Magic The Gathering Online eXchange (MtGox) server was compromised. Source code, user account details and trade data were made public.

For academic research the trade data is useful, here's a magnet to the original "public service release" file, MtGox2014Leak.zip. BEWARE: The TibanneBackOffice.zip file contains a trojan

magnet:?xt=urn:btih:b6545ecc7db8d44c8cbc4e93989edf8221af75f5&dn=2014+Mt.+Gox+Leak&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A6969&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Fopen.demonii.com%3A1337

Alternatively, cryptoanarchic.me have kindly provided an HTTP mirrored version of MtGox2014Leak.zip.

Those are the two data sources I'm using in my research.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.