Usual cause of this behavior is binding or maladjusted cruise
servo-to-throttle cable.

DS

If the vehicle operator **repeatedly** presses and releases the SET button
with their foot off of the accelerator (a "lift foot set" to begin speed
control operation), the vehicle may accelerate and exceed the desired set
speed by up to 5 MPH (8 km/h) and then decelerate to less than the desired
set speed before finally achieving the desired set speed.

The speed control has an adaptive strategy (ability to learn) that
compensates for vehicle-to-vehicle variations in speed control cable
lengths. When the speed control is set with the vehicle operators foot off
of the accelerator pedal, the speed control "thinks" there is excessive
speed control cable slack and adapts (learns). If "lift foot sets" are
continually used, the speed control overshoot/undershoot condition will
develop.

To "unlearn" the overshoot/undershoot condition, the vehicle operator has to
press and release the SET button while maintaining the desired set speed
with the accelerator pedal (not decelerating or accelerating), and then turn
the speed control switch to the OFF position **(or press the CANCEL button,
if equipped) after the speed control has been engaged for 10 seconds or
longer.** This procedure must be performed approximately 10-15 times to
completely unlearn the overshoot/undershoot condition.

My 96 and 03 vans are the same way.  They downshift with very little
added throttle.  It isn't just the cruise control, it is the
engine/transmission control programming.  The cruise doesn't make it
downshift any faster than your foot pressing the throttle.

Matt