That's entirely possible. I used to hear what sounded like gravel
hitting the door while I drove along listening to a specific CD in my
94 LS. I'd turn the radio down, trying to isolate the sound, and
wouldn't hear anything. It happened with a specific CD that had a song
on it with more bass energy than normal.

Anyway, I took the door panels off to inspect the speakers, and the
cones were ripped all the way around the perimeter of the speaker. I
replaced them with some decent speakers, and the stereo immediately
sounded much better, with no gravelly rattling noise.

The best solution for eliminating the noise without paying for new
speakers is to turn the radio off. Running it without speakers will
probably screw it up.

You can get a pair of 6" speakers to replace them for $20. Certainly not top
of the line, but similar to the originals and for my '91 teg, good enough.
The connectors aren't an exact match, but some crimping seems to have made
the connection secure without resorting to cutting wires.

As the other respondent said, my speakers were also deteriorating so that
the cones were barely connected at the edges. The bigger problem that I'm
having is with the wiring behind the radio. If I move the wire that connects
the CD player unit to the radio unit the right side will cut in and out. My
stereo is slightly newer as the original was stolen in '94.

As to step by step instructions that depends on whether you have the
hatchback or not. For my hatchback 3 screws hold the covers and then 3 more
screws come out from above to pull the speaker out. Easy.

Also you can adjust the fader control on the radio if you simply don't want
any sound from them.

--Art