There are a few things going on here:
First, if you were to create your own coin, without using any of the infrastructure created in the -coin community, you could very easily create an infinite supply of your currency and may be able to exploit it before the whole thing collapses. It is very unlikely, as currencies of any kind need to be backed by faith that it has value and will continue to have value.
Bitcoin, and other alt-coins, have a limit built into them. Without going too far into the history and future of bitcoin, basically every so often, a block of coins is "mined" that grants an amount of coins to whoever mined the block. Every so often (I believe targeted at 4 years for Bitcoin) the value of these blocks is halved. In the beginning it was 50 coins a block, and now I believe it is at 12.5 coins per block. Eventually, there will be no more coins created for these blocks. These blocks represent transactions that are occurring on the blockchain, and "miners" who are trying to solve the blocks are verifying and processing transactions. Solving the block also nets you any transaction fees in addition to the coins minted for the block (so when all coins are gone, in theory, the blocks will still be "mined" for the transaction fees).
To go to the second point, how bitcoin has value, it doesn't operate like most currencies, in that there is no bank that can just print more money, instead it operates like a finite resource. There are only so many bitcoins in existence, and there only will ever be so many. Additionally, there are some "dead" bitcoins in accounts that are not in use or have been lost (for example, there's an anecdotal story about a guy who mined 16,000 bitcoins and threw away the computer he mined them on, without having access to the private key for his wallet elsewhere. Now that bitcoins are worth $2500+, that's a lot of lost money sitting in a landfill somewhere). It's value is both in its rarity and the willingness of people to accept it as currency. As more people use bitcoin, it becomes more valuable (which is why so many bitcoin users seem like missionaries, trying to get people to convert to using bitcoin).
Bitcoin has no real value beyond it's ability to be accepted as a form of exchange. If, for example, all countries in the world outlawed the use of bitcoin, it's price would plummet and it would become worth nearly nothing. On the other hand, if a country enacted laws demanding all vendors accept bitcoin, its price would increase (as demand increases). In any case, it has no value beyond the fact people are willing to accept it as a medium of exchange. This is different from most fiat currencies, in that while most do not have any true value on their own either, most governments have sufficient power to ensure they are accepted as a means of exchange and faith in the government provides some implicit backing (plus in many countries, vendors are required by law to accept local currencies, they are not required to accept bitcoin).