How to set the absolute fee
The transaction fee is implicitly defined by the difference between the sum of input and the sum of output amounts.
fee = Σ(inputs) - Σ(outputs)
Since you are building your transactions explicitly, you can set the exact transaction fee by defining the corresponding specific amounts of the outputs for the recipient(s) and the change output. As the weight of inputs and outputs is predictable, you can calculate the appropriate fees by knowing the count and type of inputs and outputs in advance.
Why it's a bad idea
While your wallet can determine the input count during the transaction building, your users will not know in advance how many inputs it will take to fund their transaction. Since transactions get processed in order of their feerate rather than their absolute fee, the user will not be able to make an informed decision for an absolute fee without knowing the weight of the transaction. Per my experience, users get absolute fees wrong more often than not (most users even get feerates wrong), which leads to substantial support traffic with complaints about overpaying or underpaying. This is why withdrawal interfaces usually give predetermined choices (e.g. express, normal, thrifty) or take a feerate rather than an absolute fee.
If you want the recipient to pay the fee, you may want to consider using subtractfeefromamount.
