I thought the whole amount of the input is always spent.
In case the spender doesn't want to spend it whole, some of it goes to the same address (called change) returning the bitcoins.
No, if the amount in the outputs is less than the inputs being spent, the difference is converted into fee.
Software that creates transactions can explicitly add an output for change that goes back to the former owner. This can be to the same address, but generally isn't. It is called change indeed, but it's not part of the protocol. It's just how wallets deal with inputs that are too large.
What do they mean by that? Does it mean the full available amount in the input?
Exactly. It's redundant information that is already implied by referencing the transactions that created the outputs. However, it simplifies the operation of some pieces of infrastructure.
In particular, right now if you want to use a hardware signing device ("hardware wallet"), and that wallet wants to show you how much fee you'll be creating, it needs to know the amounts of the outputs being spent (as the fee is the difference between outputs and inputs). That means it needs to be given the full transactions those outputs belonged to - which can be big. Otherwise, the software that created the transaction could lie, and in fact convert a giant portion of your money into fee without you knowing it.
With the fee explicitly part of the signature hash in BIP143, if the wallet software lies about the amounts to the hardware signer, the signature will simply be invalid.