What exactly is a private key and public key in relation to Bitcoin? What are they, how are they used, and how does one find out what their private and public keys are?
Both of the keys are a part of the Elliptic Curve Digital Signature Algorithm. A private key is just a random number, while the public key is a 2D point coordinate on an Elliptic Curve derived from it. The private key is used to sign messages (in case of Bitcoin - the transactions), and the public key is used to check whether the signature is correct. The public key can either be used raw in a transaction, or turned into a Bitcoin address by means of hashing and other operations.
One can use any library that supports the ECDSA curve used by Bitcoin (secp256k1) to generate the appropriate keypair. There are some online solutions for these tasks, for example my small testing suite website.
However, most commonly one leaves key pair generation to the Bitcoin program. If needed be, the private key can be retrieved from the program by using
dumpprivkey Bitcoin API call.