Derive taproot addresses from BIP39 mnemonic

Question

I have two questions, but here's some general info to start: I wrote a little C program to generate addresses for BIP44, BIP49, and BIP84, and I wanted to (finally) update it to include BIP86. I've been trying to follow the instructions here, but for some reason it mostly isn't working. I'm using Visual Studio with libsecp256k1 from nuget (secp256k1_vc143). Unless otherwise stated, I am using the test vector mnemonic "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about".

First question: In the link, there is the "non-recommended" way to get the p2tr address, but there are no test vectors for that. I would just like to confirm that for the test vector mnemonic above the result for m/86'/0'/0'/0 index 0 should be bc1pej9yh3jd39aam30mctm8paaghg9nsemezpk0zg3udlza0nt0cy2sqvps98?

Second question: I'm trying to use libsecp256k1's secp256k1_ec_pubkey_tweak_add() function to perform the tweaking, but it doesn't seem to be working. Here is example code, where the initial private key you see is the final one when deriving the test vector mnemonic. I also hardcoded the SHA256("TapTweak")||SHA256("TapTweak") bit in tweak96.

char* seckey = "41f41d69260df4cf277826a9b65a3717e4eeddbeedf637f212ca096576479361";
uint8_t* privkey = calloc(32, 1);
HexToBin(seckey, privkey, 32);

secp256k1_context* ctx = secp256k1_context_create(SECP256K1_CONTEXT_SIGN);
secp256k1_pubkey* pubkey = malloc(sizeof(*pubkey));
size_t clen = 33;

secp256k1_ec_pubkey_create(ctx, pubkey, privkey);

uint8_t* Pb = calloc(33, 1);
secp256k1_ec_pubkey_serialize(ctx, Pb, &clen, pubkey, SECP256K1_EC_COMPRESSED);

char* tweak96 = "e80fe1639c9ca050e3af1b39c143c63e429cbceb15d940fbb5c5a1f4af57c5e9e80fe1639c9ca050e3af1b39c143c63e429cbceb15d940fbb5c5a1f4af57c5e90000000000000000000000000000000000000000000000000000000000000000";
uint8_t* tweak96B = calloc(96, 1);
HexToBin(tweak96, tweak96B, 96);
memcpy(tweak96B + 64, Pb + 1, 32);

uint8_t* t = calloc(32, 1);
SHA256(tweak96B, 96, t);

secp256k1_ec_pubkey_tweak_add(ctx, pubkey, t);

uint8_t* Qb = calloc(33, 1);
char* addr = calloc(65, 1);

secp256k1_ec_pubkey_serialize(ctx, Qb, &clen, pubkey, SECP256K1_EC_COMPRESSED);

segwit_addr_encode(addr, "bc", 1, Qb + 1, 32);

According to the test vectors and this site, I should get bc1p5cyxnuxmeuwuvkwfem96lqzszd02n6xdcjrs20cac6yqjjwudpxqkedrcr, but I get bc1pvhzh08gqauq66acu3s8a2zwyxlhexsmwqj2gey9965du0u6snwms5x4qv0. For most mnemonics I've tried, I get the wrong result, except for "warrior pudding spend capital category hospital village wait planet organ coach churn", where I get the correct answer of bc1psug2nkgfkemndxq9mwqsdxg4rvdd9s8ztysft0lpxsy0f5jmufksz0tjup and I don't know why. My code works fine for BIP44, BIP49 and BIP84, but not the tweaked BIP86. Can anyone tell me what's going wrong here?

Answer 1

For anyone that finds this question later, here's the solution:

The instructions here missed an important step: the lift_x() function as described in BIP 340. After implementing lift_x() and an ec point addition function using the mpir library, I realized I was using a really old version of libsecp256k1, so I didn't have the functions secp256k1_xonly_pubkey_from_pubkey() and secp256k1_xonly_pubkey_tweak_add(). The second one does steps 4 and 5 below. Updating libsecp256k1 at the beginning would have saved me a lot of time, so that's a facepalm. Learned a bit about ec math, so it wasn't a complete waste. Anyway, now it works.

To clarify, here is a slightly more detailed version of the steps to convert a private key to a taproot address (with taptweak):

1. Compute the public key P = privkey * G.
2. Let Pb be the serialization of P in x-only form.
3. Compute the tweak t = SHA256(SHA256("TapTweak") || SHA256("TapTweak") || Pb), interpreted as 32-byte big endian encoded integer.
4. Create a new ec point Plift where the x-coordinate is Pb, and the y-coordinate is the result of the lift_x() function
5. Compute the tweaked public key Q = Plift + (t * G).
6. Let Qb be the serialization of Q in x-only form.
7. The address is bech32m_encode(payload=Qb, version=1).