There are several definitions with overlapping meanings.
The first is perhaps best called extinct blocks. These are blocks that were produced by building on a block that is no longer the active tip of the chain. Some nodes may have considered it to be the best block at some point, but they switched to another chain which does not contain the relevant block anymore. They are valid, verified, and their ancestry up to the genesis block is fully known - they're just not currently 'active'. They are sometimes called stale blocks (typically in the context of mining software realizing it built on old data) or orphan blocks. The latter name originates from the fact that payouts from extinct blocks are denoted as "orphaned" in the reference client (referring to the fact that their coinbase transactions are now orphaned).
However, there also exist real orphan blocks, with orphan in its original meaning of "having no parent". These are blocks received by a node that does not have its entire ancestry (yet) and thus cannot be validated. Nodes keep such blocks in memory, while asking their peers to fill in the gap of their history. The client does not show these, so when people talk about orphan blocks, they are most likely referring to extinct blocks. Note that since Bitcoin Core v0.10, there are no such orphan blocks anymore, due to a significant change in the download mechanism.
Terminology is confusing here :)