I own a 2000 caravan as well and this has been a problem for me
also....like someone else said there is a tsb ,that explains the fix
but all i did was get disc, (not sure of the part number but it came
from the dealer) that bolts onto the side of the tensioner and stops
the belt from being able to slide off...worked for years now...ill
never buy another chrysler product again...what a horseshit design!
"Belt/pulley design on this van is flawed and a major safety hazard,
Chrysler should recall" but they refuse...also make sure you get a
dealer belt , any other squeaks! wish i knew the part number for the
disc..good luck!

Mike

The TSB gives part numbers for and instructs the installation of
a large stamped (for lack of a better description) washer on one
of the idler pulleys.  The part essentially acts as a fence to
keep the belt from moving off of the idler pulley.
Naturally, you need to verify that the tensioner is functioning
properly and inspect and correct the drain pipes from the
windshield wiper module to be sure they are not draining water on
top of the belt(s).
When you get the parts from the dealership, check that the bolt
they're selling you actually fits thru the stamped washer, last
one I got the bolt was wrong even though the part number was
right.  Mis-boxed perhaps?

The fix works.