In my opinion, no.

Just cut it away with a blade.  How is that difficult?  Don't use the
silicone.  Didn't your sub come with a foam gasket?

some polyfill in the box count hurt anything.
it is awsome for preventing echos...
Its dirt cheap....
go for it...

The carpeting should make a nice seal without using silicone.

"1. I'm getting ready to install a sub driver to a sub box.  I wanted to
know if I should add any poly fill or accustic stuffing.  The driver is
matched perfectly to the box size so would I gain anything by stuffing the
box lightly?"

Stuffing the box with polyfill slows down the air inside the box and will,
in essence, make the box a little larger than without.  This CAN boost the
lower frequency extension.  It'd probably be minimal and you MIGHT be able
to hear a difference.  I've used polyfill in most of my speaker boxes, but
never tried a comparison.  I would do it, but that's my personal
opinion...it's not very expensive and doesn't take much time to install.

Tony