Did you use the correct jack and also assign the jack for use with an expression pedal?
See the manual page 10. Use the Pedal Edit on the Controller screen and set it correctly.
The small potentiometer on the EV-5 is used to set the minimum value of the expression.