I've never looked for a fuse box on an escort, but these days a lot of
cars have them on the 'side' of the dashboard-  open the car door and
the fuse door is right there next to the door hinge.  You might find it
there, otherwise, maybe no.

As far as the light bulbs, if you pull them out you'll see if 1)
they're just burnt out and 2) they have a type or model number on them.
If not, you can take them to your Friendly Local Auto Parts Store and
they'll know what you need.

For the record, the backup light switch could be bad, so all the
changing of fuses and lights may not solve that minor issue.

Good luck.

-phaeton