To add onto RedGrittyBrick's answer, little endianness is a result of using little endian CPUs, and then sending data as it is stored in memory.
On little endian architectures, integers in memory are stored as little endian. When preparing messages to be sent over the network, Satoshi's serialization functions would simply copy the memory directly, thus inheriting the way that such data is stored in memory.
I believe the only exception to this is the uint256 implementation which is used for storing all 256 bit hashes. This appears to be deliberately written to interpret these integers as little endian.
Otherwise, the 0.1.0 client was written in a way that would not have been compatible with big endian architectures. If it were compiled for and ran on a big endian CPU, I think it would incorrectly deserialize integers. However the endianness used in the vast majority of computers is little endian, so this was unlikely to have caused an issue, not to mention that Windows isn't available on any big endian architectures anyways.