Anyone ever dealt with a crooked caster camber plate bearing?

From pictures that I've seen, the bearings in these things should
be straight up and down right?

I bought a set a while back and have yet to install them.  On one of
the plates that I have, the bearing is a little off to the side.

Anyone know the best way to adjust the bearings, or should I not
worry about it?

Thanks