The price of bitcoin is affected by demand and scarcity.
Scarcity decreases as more bitcoin are mined (up until the 21 million bitcoin limit is reached---this is the maximum number that can ever be mined), which decreases the price. Scarcity increases as coins are lost (e.g. through the destruction of a wallet), or just held indefinitely (e.g. the ~1 million bitcoins of Satoshi Nakamoto that 'may' never get spent). Therefore, with demand staying above the number of coins available the price will continue to increase (as people are willing to pay more to be the one able to buy the coins, beating out the others also wanting to buy them).
Now obviously if we look far ahead (assuming bitcoin survives long enough), there is the foreseeable problem that ultimately the number of bitcoins in circulation will decrease to the point that eventually one satoshi (0.00000001 BTC), the minimum value one can currently hold, will be worth more than a desired minimum, e.g. one could be worth $2000, limiting bitcoin's use to only larger transactions. Thankfully the number of decimal places in bitcoin isn't a hard limit, so this could be increased in the future such that it wouldn't matter if there were even only one bitcoin in the world.