When new orders are added to the orderbook, they are first matched against the most favorable existing orders. If the newly added order cannot be fully fulfilled, the remainder is put on the order book.
Your order would be limited to buying 600 coins and at a price of 105 satoshis. It would be filled after buying 600 coins. To have an order fulfill at a total amount to spend, you'd give a limit in the amount you want to spend rather than a number of coins you want to purchase. Defining all three, a number of coins to purchase, a maximum price you're willing to pay per unit and a maximum amount you're willing to pay is possible, but will not hit all three of your restrictions unless you get exactly the price you're aiming for.
In your example that would mean that your buy order would first match the sell order at 100 satoshis/coin, then match the 101 satoshis per coin and finally be exhausted by matching the 102 satoshis per coin. You'd have purchased 600 XXX at a total of 60600 satoshis.
If in your example the sell orders were each for 50 XXX, the resulting order book would look like this:
Before:
selling 50 XXX - 100 satoshis per coin
selling 50 XXX - 101 satoshis per coin
selling 50 XXX - 102 satoshis per coin
selling 50 XXX - 103 satoshis per coin
selling 50 XXX - 104 satoshis per coin
selling 50 XXX - 105 satoshis per coin
selling 50 XXX - 106 satoshis per coin
…
matching:
>selling 50 XXX - 100 satoshis per coin
>selling 50 XXX - 101 satoshis per coin
>selling 50 XXX - 102 satoshis per coin
>selling 50 XXX - 103 satoshis per coin
>selling 50 XXX - 104 satoshis per coin
>selling 50 XXX - 105 satoshis per coin
<buying 300 XXX - 105 satoshis per coin
buying 300 XXX - 105 satoshis per coin
selling 90 XXX - 106 satoshis per coin
…
After:
buying 300 XXX - 105 satoshis per coin
selling 90 XXX - 106 satoshis per coin
…
Also see: How do buy and sell orders work?