Let me take you down to the Strawberry Fields estate, as we follow those who deliver the monthly Labour Force Survey (LFS).

The estate is an imaginary place but there’s nothing fictive about high unemployment in working and middle class suburbs across Australia, even though the LFS statistics would tell us otherwise.

This story examines how tight definitions of unemployment and the statistical manipulation of data, drives down the unemployment rate. The media then uncritically reports these figures, allowing the government to spin them to the public, creating an enduring delusion.

Around 26,000 households are randomly selected to complete the LFS each month. That’s about 50,000 people or one in every 312 Australians, aged 15 years and over.

The ABS counts unemployed people as, “all persons aged 15 years and over who were not employed during the reference week and had actively looked for work and were available to work or were waiting to start a new job.”

People are ’employed’ if they worked one hour or more in the reference week. All OECD nations including Australia, use the same methodology and roughly the same definitions.

Most people work more than one hour per week. Even so, there is considerable conjecture amongst those working in highly tenuous and sometimes pernicious casual employment, whether such a tight definition captures their reality.

It’s also important to understand that the aim of the survey is to count employed people, not unemployed people. The unemployment rate is derived from the employment rate, so the raw number of people who respond to the survey is important.

Recipients complete the survey once a month for eight months. A new cohort replaces households that have completed eight months of surveys. So seven-eights of the month-to-month sample is the same as the previous month.

As an ABS interviewer, doorknocking in Strawberry Fields takes guts. It may shock readers to find that our Federal Government, its departments and agencies, are generally not held in high regard by the unemployed. The longer one is unemployed, the greater the antipathy or ennui.

In the vernacular, many residents don’t give a ‘fat rat’s’ for labour force data or those collecting it. The ABS states that the overall non-responder rate is about seven per cent. I suggest in areas of high unemployment and long-term unemployment, it is closer to 15 per cent.

Those who don’t respond to the survey are not listed as unemployed. According to the ABS: “Non-responding households are treated as ‘not stated’ and excluded. They are then adjusted for later in the weighting process.”

Weighting is a mathematical technique that makes the results reliably reflect variables such as unemployment, across a local population.

You can’t conduct a valid statistical survey if 15 per cent of those randomly selected have told you to ‘f-off’. Unfortunately, weighting shifts the focus from the data to ensuring a valid statistical process.

According to figures from the Macroeconomic Statistics Division of the ABS, after the initial face-to-face contact, 49 per cent of respondents complete the survey online. About 45 per cent reply over the phone, while the rest, usually the elderly, respond face-to-face.

Because the survey period lasts eight months, it often falls to Mum to log on and fill out the survey every month. That’s a low priority if you’re raising two kids on unemployment benefits. Even so, women are more likely to complete the survey than men, especially single unemployed men.

Online surveys lack compulsion and if you’re alienated, anxious and drifting to the margins of society, how likely are you to complete one survey, let alone eight of them?

I suggest the dropout rate over those eight months is predominantly the unemployed. Because they didn’t complete the survey, they are not counted in the LFS as unemployed.

There’s no suggestion of sharp practice by the ABS. It is a fine institution. This is simply how the definitions categorise the participants or non-participants.

Let us consider those unemployed living in Strawberry Fields, who completed the survey but didn’t look for a job in the weeks leading up to the survey. To be counted as ‘unemployed’ you need to be ‘actively looking for work’.

If you didn’t look for work in the four weeks leading up to the end of the survey week, you were not in the workforce. Therefore you are not counted as unemployed.

These are called ‘discouraged job seekers’. Many want to work and could start work within four weeks if offered a job, but they have suffered ongoing age or racial prejudice, they’re ill (but not on sickness benefits) or lack the schooling, training or experience to get a job. Many are the victims of previous recessions and have dropped out of the workforce, some forever.

These men and women are the ghosts of civil Australia. I estimate that unemployment is under reported by about 250,000 people but it’s hard to be definitive.

The media mercilessly attacks poor economic figures with the fury of a junkyard dog but they won’t attack the methodologies that generated those figures. Their reporting has been simplistic and uncritical. They torture numbers so they confess to anything.

Apart from a few notable exceptions, reporting on underemployment – currently more than one million people – and the rising number of ‘ghosts’ who have dropped out of the workforce altogether, has been non-existent.

Economists in the US look past the past the official unemployment rate (known as the U-3) to other metrics. One of those figures is the U-6 rate, which counts the unemployed, the underemployed and the discouraged in one figure.

It may not be statistically valid but at least it puts the battlers in Strawberry Fields on the map.

There’s currently no method in Australia to accurately count the number of people in casual work. Yet the casualisation is the most complex and pervasive feature of the labour market.

We need tools and methods that accurately reflect the dynamics of a rapidly changing labour market, otherwise we are labouring under a delusion.

Malcolm King, an Adelaide writer, works in generational change and is a regular InDaily columnist.