If you want to limit over-speeding the motor with the pot all the way clockwise, add a resistor in series with the clockwise terminal. The value of the resistor will be a small % of the pot's value. Experiment with the value that gives you an acceptable maximum speed.
Ken