On the day of the Chesham and Amersham by-election on Thursday, one voter couldn’t quite believe just who was captured on the video on their ‘Ring’ doorbell. Theresa May, the former prime minister, was door-knocking in a last-ditch attempt to get the vote out. It looked desperate and it was.

In fact, May was already well known to the electorate precisely because her face had been plastered all over election leaflets. But she wasn’t on Tory campaign material, she was quoted on Lib Dem leaflets for her opposition to the planning reforms proposed by Boris Johnson. This was a 2021 redux of the 2017 classic, ‘the Maybot wot lost it’, but in a very different way.

It was those controversial planning reforms to build more homes without local consent, plus a vehement opposition to the HS2 rail line (embodied by building works currently causing road chaos in the seat), that created an overall narrative that the Conservatives were going to rip up the Chilterns countryside.

Add in the much bigger picture of Johnson’s constant focus on the ‘red wall’ of northern seats and the Lib Dems’ spectacular success was in sending a message that this seat was sick of being “taken for granted”. In that sense, and perhaps the only sense, the Home Counties upset mirrored Labour’s Hartlepool defeat a few weeks ago.

So in many ways, there were almost textbook conditions for a Lib Dem by-election upset. The seat was so close to London, a mere Metropolitan Line tube ride away, the party could easily flood it with activists. In HS2 and planning, there were two perfectly local issues to exploit. Crucially, this was also a classic sleeper operation, a LibDem ‘quiet revolt’ with the party deliberately not shouting about its progress to avoid alerting the sleepy Tories.

All the old tricks were deployed too. One Labour figure whose mother lives in the seat confided she’d told him how nice it was the Lib Dem candidate had actually sent her a personal, hand-written letter (even though it was pro-forma, and cleverly printed to look hand-written). Getting a candidate in place early, hiding the party’s national support for HS2, all worked.

Moreover, the Tory campaign was as inept as the Lib Dems’ was impressive. One Tory MP told me how the party had been far too slow to get a candidate, had produced ‘insipid’ leaflets (“Where was the image of the PM getting his vaccine jab? Nowhere.”) and crazily focused on swing voters rather than its core vote. Despite May’s last minute doorknocks, I was told the Tories didn’t even have tellers at polling stations for large parts of the day.

As one shrewd Lib Dem old hand told me, the genius of the party’s campaign was it managed to both attract the younger more liberal voters forced out of London by expensive house prices AND the older NIMBYs who don’t want the extra housing needed to support even more of these newcomers.

And while this was in many ways a classic, localised by-election win, the Lib Dems are hoping that they can capitalise more broadly in the south. It’s certainly true that anger over planning was a factor in the party capturing other seats in the local elections (there were literal ‘disgusted of Tunbridge Wells’ vote shifts over the issue in May)

But are the tectonic plates of our politics shifting? Well, the demographic shifts of an influx of younger, graduate class to Tory seats are reflected in other seats across the south. Some in Labour firmly believe that while they should shout more about successes in places like Peterborough, West of England and Worthing in May, the fact is in parliamentary seats it’s the Lib Dems who can capitalise most.

As one Labour insider put it to me, the long term trends are good but in the short term the number of marginal winnable seats for 2023/4 in the north and midlands far outweigh the potential gains of the odd upset like Canterbury. Others in the party see it as a comforting fantasy to believe that a hipster coffee shop appearing in a Tory town somehow signals a revolution.

Labour polled its lowest ever vote share of any by-election, with just 1.6% of the vote, but those around Keir Starmer were utterly relaxed, seeing that as the inevitable squeeze of tactical voting (it’s worth pointing out that under Corbyn, Richmond Park saw a lower vote score for Labour than membership of the local Labour party, precisely because voting Lib Dem removes a Tory).

“Keir didn’t go to Amersham and Chesham, we didn’t pour resources into it. We absolutely stepped back, actively said we are not keen to engage in this. So we stepped back and allowed that squeeze message to work,” one source said. If the party does the same at the next general election in seats where the Libs are the main challenger, it would simply be repeating the Blair-Ashdown tactics of 1997.

For some around Starmer, the big, tantalising prize after Chesham is that it shows not only are voters less tied than ever to party loyalty but also that the Tories can be routed if Labour isn’t seen by southern voters as a threatening presence in No.10.

I remember joining Johnson on the campaign trail in south west London in 2015 and he correctly predicted the anti-Lib Dem landslides in seats like Ed Davey’s and Vince Cable’s, partly because of a fear that Ed Miliband would ally with Alex Salmond.

Older Lib Dem-Tory switchers are at heart liberal conservatives but they are still conservatives. As academic Paula Surridge points out today, Tory Remain voters had a tremendously low opinion of Jeremy Corbyn.

Some Starmer supporters believe that’s the main reason he can never allow  himself to be depicted as ‘Miliband-lite’, let alone ‘Corbyn-lite’. He may currently have major problems with clarity for what he stands for (which have to be solved), but Starmer’s huge asset to date has been that he doesn’t scare those southern horses.

Johnson’s two biggest problems are complacency and confusion. Today, he appeared in west Yorkshire, clearly having anticipated a cakewalk victory in Chesham that he could use to help him in Batley and Spen. The date of the Batley by-election, July 1, was clearly also designed to be a victory lap after June 21’s unlocking. So much for complacency.

As for confusion, the PM today also started talking once more about uniting and levelling up because “that’s what One nation Conservatism is all about”. That unity message was certainly his line the morning after the 2019 election, but within days he squandered it with endless culture wars on the BBC, political reporters, the judiciary, even cuts to overseas aid.

One Nationism has instead been replaced by Two Nationism, and his messages to the ‘red wall’ have often conflicted with ‘blue wall’ values. It’s not just Labour that has difficulty keeping a working class and middle class voter coalition together.

Batley (which I’ll write more about soon) is a very different seat from Chesham. Still, some Tories think one read-across is they chose their candidate too late and are still failing to target the right voters. Starmer has a brilliantly local candidate, but faces ‘headwinds’ still from the vaccine rollout and George Galloway’s targeting Asian voters. 

The biggest lesson for both Starmer and Johnson is that it’s only by trying to unify the country, not split it into different groups or regions or coloured walls, that politicians can win big majorities. In a first past the post election system, that remains more important than demographic shifts or tweaks to campaigns. Oh, and never take your base for granted.