And H.J. Kamps was sitting next to Elvis in the spaceship, which I thought
was kinda strange, but then they turned to me and said:

The needle meters out the correct amount of fuel for each type of engine
operating condition.

SU carbs have a piston that rises and falls in response to the volume of
air flowing through the carb. The needle is attached to the bottom of that
piston, and is moved in and out of the main fuel jet as the piston moves.
Since the needle is tapered, it effectively makes the opening of the jet
larger the higher the piston rises, allowing more fuel to be drawn out of
the jet as more air is drawn through the carb.

Different engines will have different air demand characteristics, so there
is a large range of needles with slightly different profiles to allow the
carb to match each engine's fuelling requirements.

If you make any modifications that improve an engine's breathing, the
profile of the needle will no longer match the engine's new air demand
characteristics, and you will need to find a needle profile that does
match it.

There's a good basic explanation (with diagrams) here:

<members.tripod.com/~RoverSD_1/carburation.html>

This is the metering needle in an SU carburettor.

Basically the inlet manifold vacuum is connected to a chamber in the
carb with a sliding piston in it.  As the manifold pressure drops, the
piston rises up further into the chamber.  The sliding piston does two
things.  It acts as an air-valve to throttle the inlet port, and at the
lower end of the piston it carries the metering needle.  This is a
tapered needle (each needle number has a different non-linear taper),
which as it rises up, exposes more of the fuel jet. (The needle fits
inside the fuel jet, and being tapered gives a jet size that effectively
varies).

It's a very clever mechanical feedback mechanism, with too many
variables to easily determine the best needle for the job!