How does Bitcoin Core's Coin Selection prevent change to be a dust output?

Question

Bitcoin Core's Coin Selection optimizes for minimal change outputs. How does Bitcoin Core prevent Change outputs of sizes below dust threshold from occurring?

Answer 1

See these lines:

// We do not move dust-change to fees, because the sender would end up paying more than requested.
// This would be against the purpose of the all-inclusive feature.
// So instead we raise the change and deduct from the recipient.
if (nSubtractFeeFromAmount > 0 && newTxOut.IsDust(::minRelayTxFee))
{
    CAmount nDust = newTxOut.GetDustThreshold(::minRelayTxFee) - newTxOut.nValue;
    newTxOut.nValue += nDust; // raise change until no more dust
    for (unsigned int i = 0; i < vecSend.size(); i++) // subtract from first recipient
    {
        if (vecSend[i].fSubtractFeeFromAmount)
        {
            txNew.vout[i].nValue -= nDust;
            if (txNew.vout[i].IsDust(::minRelayTxFee))
            {
                strFailReason = _("The transaction amount is too small to send after the fee has been deducted");
                return false;
            }
            break;
        }
    }
}

So Bitcoin core will give slightly less to the recipient rather than spend more than the payer originally requested, in the case of a dust-change output.

Answer 2

Bitcoin Core actually creates a minimum change of 0.01 BTC (if it has sufficient funds), if it can't produce a direct match.

In the rare case that it doesn't have sufficient funds to produce a non-dust change output, it will do the weird stuff Stephen mentions.