I understand that Bitcoin uses unspent tx output and Ethereum account balance based. However, recently there are lots of discussions on that Bitcoin's UTXO database has become too big. Does this mean account balance based transactions are better in scaling than UTXO based?
The choice between the UTXO model and the balance model is primarily one between privacy incentives and apparent intuitiveness.
If one follows the standard advice of not reusing addresses/outputs/scripts, as to not gratuitously reveal which coins belong to the sender and which belong to the receiver, the two models are actually equivalent. In this case balances would be single use anyway, and there would be as many balances as there would otherwise be UTXOs, removing both the apparent size advantage and convenience.
However, the balance model effectively incentivizes reuse. As the cost to the system (possibly in fees, but certainly in node operation costs) for a balance update is lower than the creation of a new balance, such a system inherently incentives revelation of transaction sources.
And even though balances are more intuitive, they do complicate reasoning about the security of the system. One issue is the problem of replay attacks. Assume X paid Y, and Y pays Z. Now W pays Y again. In a naive implementation, Z can rebroadcast his earlier transaction from Y, and could take the money from W's payment to Y. One solution is by putting sequence numbers on balances, and requiring transactions to list the sequence number of the transactions they take money of. Unfortunately, this requires maintaining the sequence number even for balances that went to zero, meaning that the balance database needs to maintain some data for every balance that was ever nonzero; a result that's far less scalable than the UTXO model was in the first place (which can forget outputs as soon as they're spent).
In short, my view is that the balance model reduces the fungibility of the currency as a whole for a short term gain that disappears.