Zone 6 to zone 3 is only a quid? Given its costs me 1.50 to go from
zone 4 to 2 I find that hard to believe.

It's true. The fares differ according to the time of day and day of
the week, and the fare in one time period (the terms "peak/off-peak"
are shunned by LU) is one pound for any journey that doesn't include
zone 1, regardless of number of zones.