It depends. Typically as a money transmitter, you are neither the owner of the transaction nor the beneficiary. You are acting as an intermediary. Whether the funds are pushed to you, or the funds are pulled by you, via authorization, the fact remains the same, you are acting as an intermediary to execute the transaction. In this case, the 'authorization' is being passed to you, voluntarily by the user by providing you with the private keys (security debate aside). If you had done this perhaps one time, you might not be considered a money transmitter. But if you do this as a business or repeatedly, even if you have no charges or a de minimis charge, you might be considered a money transmitter on a federal &/or on a state level.
Do remember, the laws are vague so as to be able to net you in, should the prosecution so decide and the government will tilt (always) with the prosecuting attorney.
When the funds traverse through you, regardless of the 'if-then-else' conditions, and these funds belong to someone else, you've stepped on murky waters.