0

I am connecting to my local bitcoind using TCP and observed the following behavior:

I see many messages where only header is given or only part of the payload is attached. Of course, it could be a bug my code too.

Are the bytes sent in chunks? Which of the following are valid?

|header1|payload1|     (as per documentation, this is valid)
|header1|              (no payload, even though required. Have seen this)
|payload1|             (no header, have seen this)
|header1|partPayload1| (incomplete payload, not sure if seen these)
|partPayload1|         (part earlier header's payload, not sure if seen these)
|header1|payload1|header2|payload2| 
|header1|payload1|header2|partPayload2|  
|header1|partPayload1|header2|
|partPayload1|header2|payload2|

Note: I am using version number 70002, relay = 1 and services = 0.

EDIT: Generally I wait for the remaining bytes, and most of the times they arrive, but in rare cases, around 2%, some other header arrives. This happens only with tx messages.

EDIT2: Based on the comments below, it seems likely that there is a bug in my code. I'll check and revert.

EDIT3: Was a bug in my code. I was assuming that partial data packets correspond to a single header. There can be multiple headers after the data is complete (in the same packet). This makes sense once I consider it as a stream.

Improve this question

1 Answer 1

Reset to default
1

TCP is a stream protocol. Even though on the wire the stream is submitted as a series of IP messages, semantically it is just a stream of bytes.

This means that at the application level you shouldn't care about the message boundaries. Routers and other internet infrastructure can arbitrarily split up data into packets to fit in their underlying protocols (e.g. Ethernet has frames of at most 1500 bytes).

Improve this answer
10
  • I am reading the data from socket and it comes incomplete (the first three cases, I have observed when connecting using TCP).
    – Jus12
    Commented Jan 9, 2018 at 11:19
  • What do you mean by incomplete? It's just a stream of bytes. If a read from the socket returns and you don't have a full Bitcoin P2P message yet, it means the rest will come later.
    – Pieter Wuille
    Commented Jan 9, 2018 at 11:21
  • The header checksum verifies, says payload should be x (non zero) bytes but the message is either only 24 bytes or less than 24+x. So I need to wait for the rest of the message, with the guarantee that the peer has actually send the complete data and its on the way, and it won't send header of a different message. Right?
    – Jus12
    Commented Jan 9, 2018 at 11:21
  • Then that means you haven't seen the entire message yet. You'll see the rest in a subsequent socket read.
    – Pieter Wuille
    Commented Jan 9, 2018 at 11:22
  • Edited my question to explain the symptoms instead of extending the comments.
    – Jus12
    Commented Jan 9, 2018 at 11:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.