If worried about the original closing transaction having too high of fees:
The signed closing transaction that you're worried about (with too high of fees) has not yet been broadcast to the network, so no harm no foul (yet).
If the two parties are cooperative, they can just agree to sign a new transaction with lower fees and broadcast that to close out the payment channel (this should be advantageous for both of them anyway).
If the two parties are NOT cooperative, it might be best to just broadcast the original closing transaction and eat the fees.
Is replace-by-fee possible for LN's closing transaction?
Replace-by-fee only applies to transactions that have already been broadcast to the network. Also, they only help if you're increasing fees, not decreasing them, so it doesn't apply to this situation.
If worried about the original closing transaction having too low of fees:
If the two parties are cooperative, you can just sign a new closing transaction with higher fees and broadcast that.
If the two parties are NOT cooperative, just broadcast the original closing transaction and hope that the transaction does not get stuck in the mempool. Also make sure that the original closing transaction has a high enough fee that there is little worry about it getting stuck in the mempool. Even somewhat non-cooperative parties should be willing to re-sign and lower the fees to close out the payment channel. Overall, this is the trickier scenario to predict.
Is replace-by-fee possible for LN's closing transaction?
Again, replace-by-fee only applies to transactions that have already been broadcast to the network. Also, this would require a cooperative party.