In a recent post I discussed the UKIP insurgency, and argued that its main source was a contradiction within the strategy of the Conservative party that began soon after New Labour’s election victory. The Conservative opposition, together with their allies in the press, increasingly focused on the alleged dangers of high immigration. In part as a result, immigration increased in importance as an electoral issue from almost nowhere in 1997 to become one of the top three issues four years later.
The contradiction was that while it was easy to talk, as William Hague did in 2001, about the UK becoming a “foreign land”, the Conservative party never had any real plans to seriously reduce levels of immigration. Focusing on high levels of immigration was a tactic for defeating a Labour government rather than a policy for government. Despite appointing a minister to the Home Office, Theresa May, who was dedicated to fulfilling pre-election promises to significantly reduce immigration, Cameron’s government found it impossible to do so without inflicting damage to the economy, something they were not prepared to do.
One of the comments on that earlier post asked why I focused on immigration, rather than talking about social conservative values more generally. It is the interaction between the social and economic that is crucial here. Immigration was powerful as a campaigning issue because it could be related to economic as well as social issues (stealing our jobs or benefits etc), and likewise doing something about it had economic consequences. In this way it is very similar to national sovereignty and Brexit, but quite different from issues like gay marriage 
Perhaps a Prime Minister and Chancellor who were more strategic and less focused on short term political tactics  could have managed a transition from an opposition campaigning to bring immigration down to a government that ignored the issue without igniting support for a more socially conservative party (UKIP). However that became impossible when the government's core policy was austerity. In that sense, the rise in UKIP was inevitable. With the threat from UKIP firmly established, it became inevitable that Cameron would make the concession that UKIP and the right wing press most wanted, which was a referendum on EU membership.
Furthermore, while it was far from inevitable that Cameron would lose that referendum, victory would not have laid the issue to rest as he hoped it would, because UKIP, the right wing press and some of his MPs would have refused to let it do so. Perhaps a lucky chance decline in immigration coupled with an improving economy and ample public service provision after a Remain victory could have reduced the popular interest in the issue, but that was not going to happen when the Conservatives’ main economic policy was to shrink the state and reduce taxes. 
This rather long prelude tells us that ending free movement was bound to be part of any Brexit deal the Conservative party would negotiate. This is clear from reading Tim Bale’s invaluable book “The Conservative Party after Brexit: turmoil and transformation”, which is an invaluable source for anyone analysing Conservative party politics from 2016 until today. It was not just that the minister tasked with reducing immigration became Prime Minister, but also any Brexit deal that allowed free movement to continue would have allowed Farage and the right wing press to undermine the party in exactly the way they did anyway, except more so.
The first part of the book deals with May’s attempts to first formulate a Brexit plan, and then try and convince the Conservative party to back it. Her plan was essentially to stay in the EU’s Customs Union but leave the Single Market. She failed to get parliament to approve that plan, time and again, and that failure led to disaster in the European Elections. Her position had become untenable, and Johnson took her place.
What is remarkable about that period is how a minority of Conservative MPs, the ironically named European Research Group (ERG), seemed to be able to dictate the form that Brexit took. Why were the majority of parliament’s MPs not able to exert their voting power and achieve a softer Brexit? To see why that was never going to work, it is easiest to imagine the following counterfactual.
Suppose that someone other than May had been elected Conservative Party leader after Cameron resigned, and that they had concluded that such a close referendum result meant that national unity should be placed above party unity. They formulated a plan that amounted to the UK leaving the EU but staying in the EU Single Market and Customs Union: a soft Brexit. They sought and got the agreement of Labour leadership and enough Labour MPs to support that plan, and parliament agreed to start the negotiation process with the EU on that basis.
During that process, the ERG, the right wing press, Farage and party members would have been in even greater uproar than we saw under May. Farage would have kept pointing out, correctly, that the Brexit vote was a vote against high immigration and this soft Brexit would do nothing to control immigration. Just as happened under May but more so, Brexit voters would have felt betrayed and deserted the Conservative party. Enough Conservative MPs, in fear of losing their jobs in 2020, would side with the ERG and threaten to depose their leader (as they have since done repeatedly). The PM, in an effort to save their job, would have been forced to revise their proposals to appease the ERG and voters.
If this counterfactual sounds pretty close to what happened under May anyway, that is because the fundamental reason why parliament could not exert its authority is the same. Too many Conservative MPs would have feared a substantial election defeat would follow any softer Brexit because of the power of Farage and the right wing press.
So just as Cameron had conceded a referendum in an effort to maintain party unity and power, the same forces ensured that we would have a hard Brexit. Anything else would have meant a strong third party on the (socially conservative) right, and therefore a Labour victory. This is one of the features of a FPTP system, which penalises having more than one party in any particular policy space.
A more favourable counterfactual to achieving her own, slightly softer Brexit was the one May herself chose, which was to get a larger majority in a General Election. If she had won that large majority, she might have been able to pass a Brexit deal that effectively stayed in the EU Customs Union. While Farage, the ERG and right wing press could attract votes away from the Conservatives because of free movement or not getting Brexit done, doing so over our ability to set tariffs would have been harder. Of course in reality she got the opposite of a larger majority.
The only other possible route I can see to a slightly softer Brexit (staying in the Customs Union but not the Single Market), other than a large May majority in 2017, is a variant of the previous counterfactual, where the new leader who put national unity above party unity and as a result had got agreement from the opposition at the beginning of the process for such a deal. Of course May did try to get agreement with Labour, but as a last resort by which time the momentum behind a second referendum was too strong. The reason why this counterfactual might have worked while something similar involving staying in the Single Market would not is that opponents of that deal might have found it hard to gain many votes on the issue of setting our own tariffs. Yet even if you believe this counterfactual was possible, you need to ask whether the Conservative party could have produced such a leader from their ranks, whether they would have been elected by their MPs and, most crucially, whether they could have disguised their intentions from the membership in the final leadership vote.
This is how under FPTP a minority can dictate to the majority. The minority need the following ingredients:
An issue that was a key reason why the party was elected, or a referendum result, but on which the party is failing to deliver (e.g. immigration,Brexit)
An insurgent party and a media that can highlight that failure and take a large number of votes from the governing party as a result
Enough MPs from the governing party who worry more about losing power (as a result of votes lost to the insurgent) than the policy issues involved, and are willing and able to side with the minority to reject their leader.
Under these conditions, the critical majority moves from MPs in parliament to MPs in the governing party, and the critical motivation moves from the issue involved to the need to retain power. Under a PR type system this possibility would be much less likely to arise, because if the governing party plus the insurgent party got the majority of votes they would have the majority of MPs, so power could be retained albeit shared. Under FPTP a majority of votes can lead to a minority of seats if the vote is split, so power is lost.
This is how the referendum result in 2016 led in just three years to the effective takeover of the Conservative party by Brexiters in 2019, and the election of a PM who anyone with any knowledge of him knew would be totally unsuited to that role. (And which was tragically demonstrated very quickly with the arrival of Covid.) The Conservative party appointed as the only person who could get them out of the Brexit hole they were in, the man who had been the person who put them in that hole in the first place. An electoral system that was supposed to marginalise the political extremes had put one in charge.  A group, as Tim Bale notes, that one rebel Tory described as “a narrow sect who wouldn’t be out of place in a Muppet version of The Handmaid’s Tale”, was allowed to take over the UK government.
 Cameron, unlike his predecessors, tried to soften his party’s socially conservative image in some of these areas.
 To quote Tim Bale from his book mentioned later in the main text: “I’m also increasingly persuaded that those individuals, and therefore parties as a whole, are as much tactical and reactive as they are strategic and proactive – actors, in other words, with incredibly short time horizons”
 As the party today looks forward to opposition, you can see the same contradictions playing out. Brexit has not led to lower immigration, because the economic costs of achieving that are too great. European migrants have been replaced by non-European migrants. While some Conservative politicians understand that there is a trade-off between getting lower taxes and lower immigration, others do not, or pretend not to for the sake of their next leadership election.
 Johnson, or those advising him, had the sense to realise that selling Brexit would require a limited and then only partial reversal of austerity. When it became clear that required large tax increases many Conservative MPs were deeply unhappy, which is one reason that Johnson was succeeded by Truss. The extent to which Johnson believed in a larger state is questionable. Tim Bale writes:
“Johnson and Sunak – the latter especially – were no different [to most Tory MPs], prepared to take action in order to prevent the economy falling of a cliff but damned if they were going to do more than the minimum required to keep so-called ‘Red Wall’ voters (and indeed voters in general) onside once the pressure was off.”
While the current leadership may be simply reaping the fruit sowed by Osborne in terms of the size of the state and the state of the public services, in terms of their attitude to civil liberties, asylum, deregulation, accountability, a pluralist democracy, privatisation, corruption, and Brexit itself, Johnson and his successors have led the most right wing administration since WWII.