Actually, there is no need to periodically query the node for the number of confirmations. The number of confirmations is implicitly given by the block height where the transaction was confirmed and the current block height.
The following assumes you store your 100,000+ addresses in a database with reasonably fast lookups, that it can also store additional data, and that you index columns that you use in queries.
Only confirmed transactions
The problem is quite simple if you only need to track transactions once they get their first confirmation. Listen for the rawblock ZMQ event, go through all transactions, and whenever a transaction sends to an address you're tracking, add this transaction to the database along with the block height where it was confirmed. Then, if you want perform some action after e.g. 3 confirmations, when a new block arrives, just look up transactions confirmed 3 blocks earlier. Depending on what it is you do, for extra robustness you might want to keep track of which confirmed transactions were already successfully acted upon.
Unconfirmed transactions
If you also want to show unconfirmed transactions, it gets more complicated as transactions might get replaced or dropped from your node's mempool. See this question on how to potentially deal with this. Whatever approach you use, the rest is the same as above.
Catching up
If your application has to restart or crashes, you might miss blocks. To guard against this, you can keep track of the last processed block height, and on startup query the node for the current block height. If you find out you're behind, just query the missing blocks (using the getblock RPC) and process them as usual.
Reorg safety
There are a few extra steps in order to make your application work correctly in the event of a chain reorganization. An approach I've found to work well is to store a few* recent block headers** (just the hash and height is enough), and when a new block arrives, follow its previous block hash (using the getblockheader RPC) until your database has the same block hash at that height as your node, then rollback any transactions made after that height. If no reorg has happened, the latest common header will just be one block behind the new chaintip, so you won't need to rollback anything.
* If you're not worried about a few tens of MB's of storage, you can just as well store all block headers.
** Storing block headers is also a convenient way of keeping track of what was the last block you processed.
