5

This is something which I think should be simple but it's driving me crazy. See for example this list of transactions associated with an address:

https://blockchain.info/address/1EZK42jGEJVniyBn1wXrUx92wzyUkYB8kJ?format=json

What I want to do is simply arrive at the same total_received, total_sent and balance stats for the given data.

An obvious requirement is to not count the "change" as a new payment to the address. My idea was to simply sum all the outputs in transactions whose inputs are not from the same target address, e.g. if a transaction has an input (or multiple inputs) from A and outputs to B and A, treat this output as change and don't include it in the sum -- only this doesn't produce the same sum as blockchain.info! In this case, it produces 158.11536395 vs 251.24164662. It looks like I'm missing something?

The next step would be to take the list of outputs produced by the first step, and see which are spent and which are not. The sum of the unspent outputs is the current balance. Is this correct?

Anyone?

  • You are along the right lines, but whether or not it's a change address is irrelevant to how much it has received/sent. – morsecoder May 1 '15 at 6:38
  • Their API docs show that you need to use the ?limit and ?offset parameters. There was pagination going on, so you weren't actually using all of the transaction data when you calculated 158.11536395 vs 251.24164662 (I got the same numbers with the data at the time of the question). – morsecoder May 5 '15 at 14:13
2

As the other answer has noted, balances don't technically exist for a specific address. This is because there may be many outputs that are addressed to an address you own, but you can only spend these outputs discretely (I.e. You can say deduct x from my account, you can only say deduct this output amount which was previously awarded to this address).

We can define a pseudo-notion of a balance, though, let's call it the address' unlock-able value (UV), because the owner of the private key for that address can unlock some # of coins by knowing the private key for that address.

Now, to calculate the UV balance, total received, and total sent of an address, you follow this pseudo-algorithm:

Address a;
Uv = 0;
Received = 0;
Sent = 0;
For each (output in blockchain):
    If (output is unlock-able by owner of address a):
        Received += output.amount;
        If (output has been spent):
            Sent += output.amount;
        Else:
            Uv += output.amount;

At the end of all the outputs in the blockchain, the Uv amount will have a value that is similar in concept to a balance. But here we can also see why this is a very tricky notion:

  • How do you know if an output is unlock-able by owner of address? Sometimes only the key holder themselves can know.
  • What if an output can be unlocked by, say, 1 of a set of 3 keys? Do you credit the balances of all three keys, or none?

I suspect the way blockchain.info does this is by basically running the above pseudo algorithm (although much more optimized and using a database), but only considering outputs in the Pay-to-Pubkey-Hash format.


Alright, through contacting the blockchain.info support team and some testing I've found that the problem was the API link given in the question is both paginated and rate limited. The following PHP code produces the same values that blockchain.info shows on their site, assuming you don't run into rate limiting problems.

<?php

// Method: POST, PUT, GET etc
// Data: array("param" => "value") ==> index.php?param=value

function CallAPI($method, $url, $data = false)
{
    $curl = curl_init();

    switch ($method)
    {
        case "POST":
            curl_setopt($curl, CURLOPT_POST, 1);

            if ($data)
                curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
            break;
        case "PUT":
            curl_setopt($curl, CURLOPT_PUT, 1);
            break;
        default:
            if ($data)
                $url = sprintf("%s?%s", $url, http_build_query($data));
    }

    // Optional Authentication:
    curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
    // curl_setopt($curl, CURLOPT_USERPWD, "username:password");

    curl_setopt($curl, CURLOPT_URL, $url);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

    $result = curl_exec($curl);

    curl_close($curl);

    return $result;
}

$address = "1EZK42jGEJVniyBn1wXrUx92wzyUkYB8kJ";
$offset=0;
$txs = array();

while (true)
{
    $val = json_decode(CallAPI("GET", "https://blockchain.info/address/$address?format=json&limit=10&offset=$offset"), true);
    if (count($val["txs"]) == 0)
        break;
    else 
        $txs = array_merge($txs, $val["txs"]);

    $offset += 10;
}

$uv       = 0;
$received = 0;
$sent     = 0;
$nrefs    = 0;

foreach ($txs as $tx)
{   
    $ininputs = false;
    $totalin = 0;
    foreach ($tx["inputs"] as $input)
    {
        if ($input["prev_out"]["addr"] == $address)
        {
            $ininputs = true;
            $nrefs += 1;
            $totalin += $input["prev_out"]["value"];
        }
    }

    $inoutputs = false;
    $totalout = 0;
    foreach ($tx["out"] as $output)
    {
        if ($output["addr"] == $address)
        {
            $inoutputs = true;
            $nrefs += 1;
            $totalout += $output["value"];
        }
    }

    $uv       += ($totalout - $totalin);

    // $received += $totalout;
    // $sent     += $totalin;

    if ($ininputs && $inoutputs && $totalout >= $totalin)
    {
        $received += $totalout - $totalin;
    }
    else if ($ininputs && $inoutputs && $totalout < $totalin)
    {
        $sent += $totalin - $totalout;
    }
    else if ($inoutputs)
    {
        $received += $totalout;
    }
    else if ($ininputs)
    {
        $sent += $totalin;
    }
}
echo "\n";

echo "nrefs    : " . $nrefs . "\n";
echo "uv       : " . $uv . "\n";
echo "received : " . $received . "\n";
echo "sent     : " . $sent . "\n";

Result of running program

enter image description here

  • I don't think your algorithm is complete. For one, it double counts "change" for payments which goes back to the original address (e.g. address A spends outputs and pays to address B and the change goes back to address A, where it gets counted as it were originally given to it, i.e. it's counted twice: once in the original outputs, and once again as change). This is the gist of my original question: you can't run a simple sum like your own example, so what can you do? – Ivan Voras May 1 '15 at 19:56
  • Hmm, so you're saying that you don't want to double-count an address sending coins to itself towards the total received? But what if two addresses send coins back and forth to the other, even if both owned by the same user, is that really any different? You can only go by what's in the blockchain, and if the blockchain says that an address was 'spent from' and awarded coins in the same transaction, then that's what happened. – morsecoder May 1 '15 at 21:04
  • Trying to infer real world information will only lead you down a rabbit whole where you can be sure you won't cover all the edge cases. Best you can do is go by the blockchain. Now, it might be possible to calculate total unique coins an address has ever owned, adjusting for the mixing that happens in a transaction. While that would be an interesting number to calculate, I don't think it makes the best sense for the 'total received' number. See this for more info on tracking % of coins. – morsecoder May 1 '15 at 21:05
  • Well, basically, I need to do whatever blockchain.info does, and it looks like one of the things it does is not double-counting change in total received. It looks like it's doing even more, which I'm hoping to find out. – Ivan Voras May 2 '15 at 12:02
  • @IvanVoras, see updated answer. – morsecoder May 5 '15 at 15:34
1

The sum of the unspent outputs is the current balance.

Not really, that's just the unspent outputs for a particular pubkeyhash. Balances don't exist in Bitcoin, it's extremely misleading to people to suggest that they do by using this terminology.

What I want to do is simply arrive at the same total_received, total_sent and balance stats for the given data.

It's not a part of the network so it's not even defined what this term could possibly mean. The information displayed isn't consistent between the websites claiming to be able to show a "total received" for an address.

So who is right?

It's better just not to confuse everyone by showing faulty data to begin with. Extrapolating this information out to people doesn't do any good, and if services are built using these sort of faulty assumptions about the behavior of the network it can be potentially disastrous and lead to loss of funds.

  • 2
    For better or for worse, people in the real world think in terms of account "balance", for their and other people's accounts, so I don't think that is changeable. What do you suggest as an alternative name? "Spendable amount"? For starts, I'd like to replicate some or any of the non-trivial results (521.29736505 is just a plain sum of outputs, including change). Any ideas on how blockchain.info does it? – Ivan Voras Apr 30 '15 at 21:52
  • Total spendable outputs would be better, but it would be better if this sort of stuff wasn't shown to end users to begin with. I don't think anybody but blockchain.info could tell you how they calculate theirs. – Anonymous Apr 30 '15 at 22:16
  • 3
    Balances don't exist in Bitcoin, That might be true at a protocol level, but people don't directly construct Bitcoin transactions - they use clients that abstract those details away. I mean, you could apply similar logic to say that Domain Names don't exist, since you're really communicating with an IP address. – Nick ODell Apr 30 '15 at 22:17
  • A wallet can have a balance, but an address most certainly does not, it has a collection of atomic outputs that can be either spent or unspent. A "balance" implies many things that are just simply not applicable. As for the results above, I don't think anybody actually understands how anybody else is calculating this, which would explain the variance. – Anonymous Apr 30 '15 at 22:18
  • 3
    @Bitcoin A "balance" implies many things that are just simply not applicable. And saying that you open your web browser to a domain name also implies things that aren't true. And yet, it's a useful abstraction.I don't think anybody actually understands how anybody else is calculating this, True for total_received, but balance is much better defined. – Nick ODell Apr 30 '15 at 22:21
0

Some of these answers are absurd. Yes, there is a difference between having the key to spend from an address and simply totalling in and out. But if you have the key pair to spend from an address, its final total is precisely exactly a 'spendable balance'.

I have no idea how the block explorers lack a consensus on account balances. I would guess likely the bigger numbers are accurate sums of an address in/out. The inaccurate ones would be lower and are inaccurate probably because they don't compute the coinbase properly for each input that ends up in an address and isn't spent.

Coinbase values are not stored in the chain, they are in the chain parameters, what you might call 'ephemeral consensus'. Coinbase values are calulated according to the block reward halving schedule. Getting a different total at the end can only be because you are not counting all of the valid inputs and outputs, of which coinbase is very very important.

  • Coinbase values are not stored in the chain, they are in the chain parameters. This is incorrect, the current block reward + fees is very much on chain, in the output of the coinbase transaction. Any block explorer summing up outputs should be able to pick it up without additional work. – Raghav Sood Oct 16 '18 at 13:23
  • Do you mean is that you can infer the coinbase value by the total value of outputs it used as an input? – Louki Sumirniy Oct 16 '18 at 13:29
  • But you can't say what the next block's coinbase value should be based on the data in the chain. Strictly speaking although a precise number of blocks relates will be found to have a consistent coinbase value, the halving regime, ie, number of blocks, initial subsidy, it is NOT coded, though the result is. The function GetBlockValue and the start and min subsidy values in the source code dictate the block reward. Like checkpoints... It's what I am calling ephemeral consensus, and any node ignorant of it will fork the chain. – Louki Sumirniy Oct 16 '18 at 13:52
  • Sure, but you can predict the reward based on the height. Moreover, the current reward is irrelevant when calculating the total sum of inputs and outputs for an address, since all the information you need for that is on the chain. The current block reward has no bearing on that. – Raghav Sood Oct 16 '18 at 19:32
  • The current block reward does matter for generation transactions. However as I learned today the 'value' field tells you anyway and if it's the only tx in a block it's quite clear. But you can only infer from more than halving period number of blocks when it occurs, and it's an edge case, sure, but getting that wrong would lose you a block reward. – Louki Sumirniy Oct 17 '18 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.