I would like to know in which order the leveldb iterator iterates over the chainstate entries. I aim to serialize the chainstate and have to make sure that every node can calculate exactly the same serialization (to compare its hash afterwards).
The LevelDB iterator iterates through the keys in lexicographic order. From the docs:
The preceding examples used the default ordering function for key, which orders bytes lexicographically. You can however supply a custom comparator when opening a database. For example, suppose each database key consists of two numbers and we should sort by the first number, breaking ties by the second number.
However, I am not sure that relying on the iteration order of one particular DB is what you should be using, since every other implementation would have to reproduce that iteration order for compatibility. So you would need to require a canonical ordering anyway, which can be implemented using comparators.