I assume the scriptPubKey is contained in a UTXO, and therefore embedded in the blockchain. Since, this is the script used to unlock the input.
Yes, absolutely.
Is a scriptSig also embedded in the blockchain?
They're part of transactions, and transactions in their entirety are part of the blockchain. All data that is directly or indirectly needed to validate blocks must be part of the blockchain, or the network wouldn't necessarily come to an agreement.
For example, if scriptSigs were not part of the chain, you could create a valid and invalid version of a transaction, broadcast the valid one, but then once you saw it in a block, replace it with the invalid one. Now the node you're passing it onto would think that block was invalid, while others thought it was valid - leading to a permanent fork in the chain.
Or is it only held in memory by miners to verify signatures before confirming a block, then discarded?
That's an implementation detail, and completely independent from the rest your question.
Yes, some nodes only download (all) blocks, validate them fully, and then throw them away after validation ("pruning"). Those blocks and everything in them are still part of the blockchain - these nodes simply don't store the full blockchain.