I think there's a natural drift to the right as a country becomes richer; to modern eyes even Callaghan qualifies as *wildly* left wing, even if he seemed centerist at the time.
These days they seem to get treated like football managers - no matter how good a job they do it only takes a couple of lost matches to see them turfed out and replaced with someone worse...