The leader of the Labour party is making us a cup of coffee. This is not any old instant; this is a beverage from an expensive silver machine which, he explains, was a gift from him to his coffee-fussy wife. As he manipulates the controls, barista Miliband apologises that the process is laborious. Time passes – quite a lot of time – to the sound of beans being ground. Then there is much gurgling and whooshing and frothing. This could, we suggest, be a metaphor for how long it is taking his party to produce any big, voter-grabbing policies. A cloud of anxiety scuds across his face, then he brightens: "I promise you it will be worth the wait." When it finally materialises, the product is quality – the coffee that is.

We will talk about whether the policies will also be worth the wait in a moment, but first we discuss the turbulent build-up to the Labour conference in Brighton. It cannot have been what he hoped for. Critics, friendly and not friendly at all, have been pouring what one colleague calls "10 buckets of shit" over the head of the Labour leader.

He shrugs. "You get ups and downs. You've got to learn to take the rough with the smooth. It comes with the territory of being leader of the opposition. I'm totally unfazed by it all."

This is zen Miliband speaking, a man preternaturally tranquil, or at least highly skilled at giving the impression of being so, in the face of a wilting poll lead, poor personal ratings, savage attacks on his character from the Tories and their media, and searing critiques from within his own party. His calmness in the face of adversity impresses some friends. To others, it is an anxiety-inducing indication that the Labour leader still has not grasped just how viciously personal the next election campaign could be. He half acknowledges that he understands what he might face from the Conservatives. "By any means necessary. That is their modus operandi, isn't it? By any means necessary."

Complaining about how the Tories operate has just been made a whole lot harder for Labour by the publication of the confessions of Damian McBride, serialised in the Daily Mail under the headline "Poison at the heart of Labour", in which Gordon Brown's most notorious hatchet man details his brutal destruction of the reputations of Labour ministers. Surely Miliband, a senior member of the Brown inner circle, knew what McBride was doing? "I complained to Gordon about what Damian was up to," he says. "I was worried by him and I said to Gordon I was worried by him."

He wants to dismiss this as "ancient history", but agrees that there is a warning to be heeded from the vicious feuding which disfigured the last Labour government. "The lesson we should draw is no factions, no briefing. If you think about the way I've led in the last three years, we don't have briefings against people in the shadow cabinet. We don't have that kind of thing happening."

So McBride will never be allowed to work for the Labour party again in any shape or form? "No." Given the Labour party's unhappy history of internal warfare, it is a bold claim to say there will never be factionalism again. What will happen to anyone caught briefing against colleagues while he is leader? Will he sack them? "Yes. Totally unacceptable."

From time to time, stories of tensions between him and Ed Balls bubble up to the surface. He observes that some have predicted that the two of them were "going to be Blair/Brown all over again". In the past, he hasn't always been unequivocal about whether Balls will be chancellor in a Labour government. So it is striking that he offers an especially warm endorsement now.

He says, in terms that leave no wriggle room, that the other Ed will remain as shadow chancellor all the way to the next election. He goes on: "Let me say something about this that is really important. Nobody doubts his ability to do the job of chancellor. He's somebody with the economic knowledge, the economic judgment. We work incredibly closely together."

For some internal critics, Miliband has been obsessed with keeping the peace to a fault. As a result, too many issues have been left unresolved and too little has been produced by way of solid offers to the electorate. He makes the argument for strategic patience. The first three years of his leadership were about laying "the intellectual foundations". This is the year when his consistent, but sometimes rather abstract, themes – "responsible capitalism", "pre-distribution", "the squeezed middle" and "one nation"– will "come together" and he "will show what it means in practice for people".

The "core of my argument" will be that "the cost of living crisis" is the biggest issue in British politics. "Somewhere along the way, the link between the growing wealth of the economy and family finances has been broken."

Miliband believes that the Tories have made a strategic error by claiming that they have put the economy back on track. "Some people say: oh well, the Tories have been proved right on the economy. It is nonsense. The idea that David Cameron and George Osborne are going on a victory lap! We've got the slowest recovery in 300 years, we've got a million young people looking for work, we've got more people unemployed for longer than at any time in a generation, we've got the longest fall in living standards since 1870. The idea that this is cause for celebration and, you know, taking your shirt off and flinging it into the crowd – which is what they seem to be doing – I mean it's absolutely extraordinary."

It is "totally jarring with the public", who see the Tories "on the victory lap" while their "lives are getting harder, not easier". When David Cameron talks about the global race, Miliband contends, what the Tory leader means is a "race to the bottom" – in wages, jobs and rights at work.

So what would he do? "You've got to prevent all of the gains being scooped up by those at the top." Wages are "a key part of it" so that those on middling and low incomes have more money in their pockets. A variety of policies to address the squeeze on living standards will be announced at the conference. To us, he unveils a series of measures designed to strengthen the minimum wage. A Labour government would improve its real value "because it's fallen back over time during the recession". Labour is also looking at the case for higher minimum wages in different sectors – a not uncontroversial idea which could mean a higher minimum wage in, say, banking and a lower one in retail.

He also reveals plans to be much more aggressive towards employers who break the law by not paying the minimum wage. It has been estimated, he says, that a third of workers in the care sector are not getting it. He will boost the current "hopeless" levels of enforcement – just two prosecutions to date – by giving local authorities new powers. Fines will be dramatically increased.

"At the moment, if you don't pay the minimum wage, the maximum fine is £5,000. If you engage in fly-tipping, the maximum fine is £50,000. That is ridiculous. If you engage in systematic abuse of the minimum wage, you should have a maximum fine of £50,000," he says.

That ought to be pleasing to the ears of many workers. It also ought to sound good to the union general secretaries, some of whom are otherwise very angry with Miliband for trying to reform the Labour-union link. He started this battle rather late in the parliament, but he says he is determined to prevail. Having spent a lot of his leadership distancing himself from New Labour, in this case he seizes on praise from Tony Blair that this reform is "transformational, revolutionary".

Miliband says he is trying to do the unions a favour. "My starting point is not that the voices of individual working people are heard too loudly, but they're heard not enough. This has been the complaint of many people in the unions over 20 years: why aren't you rooted more in the community?"

He mentions his own Doncaster constituency, where once members of the miners' union would also be Labour members. "It doesn't happen in that way any more."

Giving individual trades unionists a direct relationship with the party, rather than one channelled through and controlled by a few general secretaries, will make it more not less likely that a Labour government will heed the concerns of working people. "It is a change which is hard for many people in the unions, but it will also put an additional obligation on the Labour party. Part of our finances will depend on reaching out to these individual working people and saying it's worth being part of our movement. I am optimistic enough to believe that you can have a party not of 200,000, but 500,000 or 600,000. It will take time, it will be difficult, but it will rely on us reaching out to people."

Only once has Labour returned to government after just a single term of opposition. As Miliband says himself, he must "defy history" to win the next election. "Is it going to be a tough fight? Yes, it is. But I'm absolutely confident."

Thanks to the pro-Labour bias in the electoral system, it may be mathematically possible for him to become prime minister with about 35% of the vote. If you could offer that now as the result of the next election to his more pessimistic colleagues, they would take it. Would that be good enough for him?

"No," he says emphatically. "No. I have much more significant ambitions than that in terms of who we get to vote Labour. I want to do better."

This believer in slow-brewing politics has just 20 months left to prove that he can.