I doubt you did any damage during just a few times.  If you start to feel
slipping, especially if your rear end starts to slide to one side, then put
it in 4x4.  Also gentle turns on icy/snow covered roads you may want to keep
it in 4x4 and drive at reduced speeds for both the drive train and to
maintain control.  It's the tight turns (city driving at an intersection for
instance) on cleared roads that will cause you grief.  I don't live in snow
country, but sometimes visit there, and often drive off pavement and shift
into and out of 4x4 frequently as conditions merit.   Also, shallow snow can
provide very good traction and 2x4 works well.

One of the rare times it snowed in the mountains near my house (once or
twice every 5 years or so) I went up the next night.  A group of light 4x4s
with little experience were trying to go up a section that had been turned
to slush during the day and re-froze by the time I got there.  None of them
could make it.  Fortunately, my truck had enough traction with 4x4, limited
slip rear differential and lots of weight to make it past the ice. Once I
got around the corner where they couldn't see, I got off the ice and up on
to the snow about 4"-6" deep and experienced very slight drive line binding
in corners.  I drove the rest of the way in 2x4.

Anyway, you probably have much more experience driving in snow and ice, but
I have 20+ years driving 4x4s (mostly desert and mud).  Because the transfer
case ties the front and rear axles together, if one wheel has good traction
and another poor traction, when you use the brakes, the wheel with good
traction will pull the poor traction wheel causing it to skid which in turn
decreases it's traction even further, hence increasing your stopping
distance and/or causing the vehicle to turn sideways.  That's one reason I
drive slower in 4x4, the other being that the manuals for my 4x4s list a top
speed of 50MPH or there abouts.  Also, because of the differentials the left
front and right rear wheels (and vice-a-versa) are tied together.  If you
loose traction on the tires at opposite corners of your vehicle, you could
get stuck which means you want to go faster when you foresee that condition
to get past it and get good traction back to the tires.  All-in-all 4x4s
have a learning curve that isn't difficult to overcome and it's often fun.

If the roads are _really_ slippery you won't have much trouble cornering
in 4x4mode.  
At some stage  you'll notice the car slipping although the road is not
very wet or icey. I made the mistake of using 4x4 anyway and noticed I
was needing 4x4 more and more...  
Yes, the tire tread had finally worn out.. (after 6 years)    :-/

No, but just don't force it. Sometimes one forgets one is still in 4x4
in the city until you go for the parking space....then you remember!
You'll need 4L very occasionally eg deep mud, steep slope etc.