Neither fiat currencies nor Bitcoin is backed by anything tangible - you cannot through any mechanism expect to be able to exchange them for a fixed quantity of any other good or combination of goods.
Neither is there any agreement associated with fiat or Bitcoin to value anything by the amount of labour it took to produce it.
Bitcoin and fiat currencies have a lot in common when it comes to what gives them value; properties that make them suitable as medium exchange, (possibly) store of value and (possibly) unit of accounts: durability, low transaction costs, irreversibility (for fiat only some transactions are irreversible, mainly cash), security, and scarcity.
The 'math problems' you refer to are hash computations. They don't have value outside of the Bitcoin network, but they add value to the currency because they make the Bitcoin system secure against double-spending (someone trying to spend the same money more than once). Just like when you accept fiat currency notes you do not get any value specifically from the watermarks, but the application of watermarks add security to the currency - and therefore makes it more valuable to people.
Neither Bitcoin nor fiat currencies would have any value, however, were it not for a coordination of people who agree to use them as money. For fiat currencies this coordination happens through a government monopoly on issuing such money, and insistence that taxes are paid using the monopoly fiat currency.
For bitcoin this coordination has happened because it was a first mover in the technological innovation that it represents: bringing about lower transaction fees (irrespective of geographical distance), faster payments, irreversibility, pseudonomity, and very importantly a lack of reliance on any central party (it is distributed).
Bitcoin will only be able to sustain this coordination (and its value) if there are significant network effects, which makes it difficult for new entrants in the cryptocurrency market to gain market share, and there is no technological innovation which makes Bitcoin inferior to a new payment mechanism.
The initial and current value of Bitcoin, starting from zero as it did, is based on people recognising the above features and also that the scarcity of the coin could make it appreciate in value because supply is limited and at some stage effectivley stops growing.