As described in BIP-113. Wouldn't the last 12 blocks bring us closer to a one-hour offset (ie 120min, midpoint 60)?
BIP 113's goal is not to aim for a specific offset.
Its goal is guaranteeing monotonicity (treating every block's timestamp as strictly larger than the timestamp of each of its ancestors). It does this by leveraging the existing consensus rule which states that the median of the timestamp of a block has to be strictly larger than the median of its 11 predecessors.
As to why the number 11 eleven was chosen in that existing consensus rule, we don't know. It was chosen by Bitcoin's creator.