The Life Project: what makes some people happy, healthy and successful and others not?

The factors that most affect our life chances are revealed as the first group of British babies followed in a remarkable cradle-to-grave turns 70

In March 1946, scientists recorded the birth of almost every British baby born in one, cold week. They have been following thousands of them ever since, in what has been become the longest running major study of human development in the world. These people - who turn 70 over the next two weeks - are some of the best studied people on the planet. And the analysis of them was so successful that researchers repeated the exercise, starting to follow thousands of babies born in 1958, 1970, the early 1990s and at the turn of the millennium. Altogether, more than 70,000 people across five generations have been enrolled in these "birth cohort" studies. No other country in the world is tracking generations of people in quite this way: the studies have become the envy of scientists around the world, a jewel in the crown of British science, and yet, beyond the circle of dedicated researchers who run them, remarkably few people know that they even exist.

I have spent the last five years researching these studies, and carried out well over 150 interviews with scientists, science administrators and cohort members along the way. (The identity of people in the studies is confidential, but I was able to talk to a few.) I discovered that this is a gloriously British endeavor - run by a cast of eccentric English men and women, often on a wing and a prayer. I came to believe that remarkable things happen when scientists do something as simple as watch people live their lives, and try to work out why we follow different paths.

The birth cohort studies have amassed mountains of information - including rooms stuffed with paper questionnaires, terabytes of computer data, freezers full of DNA, and boxes packed with fingernails, baby teeth and slices of umbilical cords, all carefully preserved. There is even a secure storage barn in Bristol containing around 9,000 placentas, pickled in plastic buckets. Together, these records chart the lives of ordinary British people in painstaking detail as they have lived through the tumultuous decades since the war. The findings from them have been both prolific and far-reaching, generating more than 6,000 academic papers and books. They have fed into policies regarding pregnancy, birth, schooling, social mobility, adult education and more, and have shaped scientists' understanding of issues ranging through foetal development, chronic disease, ageing and death. They have touched the lives of almost every person in Britain today.

Yet often, the observations that scientists have made through these studies have not made for comfortable reading: they have revealed the persistent inequalities in society, and how the obesity epidemic has hit us hard. As one scientist told me, the birth cohorts hold a mirror up to Britain, and sometimes we don't like what we see.

1940s: birth

The whole enterprise sounded utterly implausible back in 1946, when scientists dispatched health visitors to interview every woman who had given birth in one week. No one had ever attempted to collect detailed information on such a large group of mothers and babies before. But, somehow, they pulled it off. Were you able to get your full extra ration of a pint of milk a day, the mothers were asked. Who looked after your husband while you were in bed with this baby? How much did you spend on vests, petticoats, matinee coats, bootees, bonnets, shawls and rubber sheets for baby? And how much did you spend on smocks, corsets, brassieres and knickers for yourself? (Although the aim of the survey was to understand Britain's falling fertility, it is notable that the mothers were asked nothing about sex.)

The results, when they emerged, shocked the nation by revealing the dismal conditions in which working-class women were giving birth: the babies in the lowest class were 70% more likely to be born dead than those in the most prosperous, and working-class mothers received worse medical care - for the obvious reason that they couldn't afford it. But the results made a splash, appearing just in time to be integrated into plans for the NHS: when it launched, in 1948, the medical care associated with pregnancy and birth became free, and around that time more generous maternity allowances were introduced. The report "was a blueprint for the maternity service we've had ever since", one obstetrician later said. So the survey helped to create the lasting belief that pregnant women deserved support by the state, which over time has grown into the maternity leave and benefits that families receive today.

1960s: school

The scientist then focused on the next big challenge that life throws at us: school. The first cohort study offered a powerful way to test the success of the 1944 Education Act, which introduced a tripartite system of schools : grammar for the brightest, followed by secondary modern and technical schools. Students were streamed into grammars based on their performance in the 11-plus exam, the idea being that the cleverest children would pass the exam, regardless of background and social class. But did they? Had class ceased to matter; were brains all it took to get ahead?

Unfortunately, no. The study revealed that bright children from the working classes were far less likely to do well at school and pass the 11-plus than equally bright middle-and upper-class children. The attrition of smart but poor children became known as the "waste of talent" and the outcry quickly turned the key book of results - The Home and the School - into a must-read educational reference for student teachers. Many remember it still. "It is hard to imagine a rival in fascination and importance," read one review in the paper.

In 1965, the year after the book was published, the Labor government drove through a major expansion of comprehensive schools, designed to replace the selective school system with a one-size-fits-all approach. So the cohort study selective or comprehensive education serves children better rages almost as fiercely now as it did nearly 50 years ago.

1970s : smoking

It is hard to believe that even in the early 1970s, smoking in pregnancy wasn't given much thought: 40% of pregnant women smoked. Luckily, the birth cohort studies turned things around. Back at the start, just as the second birth survey was going to the printers, the founder of the study raced in and told the team to stop so that he could add a question about whether pregnant women had smoked. The delay would prove invaluable: the scientists gathered detailed information on women's smoking habits, and counted up all the stillbirths and infant deaths.

Later, when they crunched the numbers, they showed convincingly that women who smoked during pregnancy tended to have babies of lower birth weight, and that this increased the rate of perinatal death - even when confounding factors such as social class were taken into account. When their meticulously argued paper appeared in the British Medical Journal in April 1972, many doctors and scientists were finally convinced. "No reasonable doubt now remains that smoking in pregnancy has adverse effects on the developing foetus," stated an editorial in the BMJ at the time. The finding changed public health advice, helped establish the idea that smoking in pregnancy is dangerous and has been saving lives ever since.

1980s : obesity

The power of cohort studies of people born at different times is that it allows generations to be compared - and that is how scientists were able to watch us all gaining weight.

Hardly any members of the first cohort - the postwar generation - were overweight as children because food was rationed, and they stayed healthy weight through young adulthood too. But in the 1980s, as the cohort entered their 40s, the proportion of people who were overweight or obese began to rise and it has been going up ever since. Just last year, scientists compared the way that people in all five generations have gained weight during their lives. This study was gargantuan, including 273,843 measurements of body mass index on 56,632 cohort members. No one has previously been able to watch obesity as it crept up on consecutive generations, and it didn't make a pretty sight.

The scientists found that obesity rose in the first three cohorts in the 1980s: they all started to become overweight at around the same time, even thought they were different ages. Why? Researchers suspect the answer has to lie in our environment and lifestyle, which changed enormously in the 1980s, when incomes in the UK were climbing, eating out was more affordable and cars were the way to get around. People who were more susceptible to weight gain started to pile on the pounds when our lives underwent this radical change.

As for the later generations, the wave of obesity had already wept in by the time they were born: the data shows that they were more likely to be overweight or obese as children. Studies of the millennium-born children showed that 23% of children were either overweight or obese by the age of just three, and a similar percentage by the age of five. On the positive side, not everyone gains weight and there is now a focus on understanding what special combination of genes and lifestyles makes some people seemingly immune to all the pressures around us to pile on the pounds.

1990s: adult education

In the early 1990s, scientists started putting some study members through a string of tests. On what page if the Yellow Pages are the plumbers listed? How much change will you have from 2 pounds if you buy a 68p loaf of bread and two tins of soup for 45p each? What is the floor area of a room that is 21 x 14 feet? The answers to these somewhat arbitrary-sounding questions revealed another uncomfortable truth about modern Britain: a large proportion of adults struggle with reading and maths.

The studies showed that about one in five adults had a reading level lower than that expected of an 11-year-old child. When it came to numbers, things looked ever worse: one in three adults had skills below those expected of an 11-year-old (they could not calculate the area of the room, even with a calculator) and a quarter had skills at or below the expected level of a seven-to nine-year-old (they could not work out the change from the 2 pounds problem). A comparison of Britain's basic skills with those in other countries in the OECD showed that Britain was loitering near the bottom of the league. This widespread lack of basic skills had gone largely unnoticed until the cohorts brought it to light.

The studies were a big driver behind a major adult education initiative introduced in the early 2000s, a raft of measures to boost adult literacy and numeracy, including free adult courses to those without the equivalent of a satisfactory grade at GCSE English or maths. Several million people eventually signed up, and follow-up suggested that they often emerged from the courses with higher motivation and self-esteem.

2000s : social mobility

One thing about cohort studies is that you know where they start, but you never know where they will end up. So when scientists recorded the births of thousands of babies in 1958, no one imagined their efforts would shape government policy on life changes 50 years down the line. But that is exactly what happened, when work carried out by economists in the 2000s suggested that social mobility had declined.

The aim of the work was to examine the income of parents, and then the income of their children, to find out if the same families were still at the bottom of the heap. The results came as a shock: showed that the income of children born in 1970 was tied more tightly to the income of their parents than it was for those born in 1958. It had become harder for children born in 1970 to escape their background : poverty, it seems, had become a stickier glue. The finding inspired a major academic debate - some scientists question it still - and swept into politics too. The idea that social mobility is declining was very easy to digest and it landed at an opportune time, right in the middle of a growing debate about inequality. It helped drive today's political focus on improving social mobility and equaling life chances.

But that is going to be tough: the cohorts show that inequality has dogged every generation. In every cohort, children born into disadvantage have tended to follow a more difficult trajectory: they are more likely to struggle at school and at getting jobs, and to suffer poor health, among other things. But the studies have circumstances ends up in them : there are routes to escape.

Evidence from these studies and elsewhere has pointed to the enormous value of parental interest and involvement in children, particularly in the first few years, and suggested that interested parents might compensate for (but not eliminate) the disadvantage concomitant with a difficult start. In 2006, a cohort report called Bucking the Trend examined what enabled disadvantaged children born in 1970 to succeed in education later in life. It concluded that parental engagement, particularly in the first few years of life, is more crucial than anything else. For example, children whose parents had read to them when they were five and showed an interest in their education at age 10 were significantly less like to be in poverty at age 30.

2010s: getting old

As the oldest of the members reach 70, the scientists have the unenviable task of tracking them to the grave. (The participants are unruffled by this. Most say that being in the study makes them feel special, and that it is their duty to see it through to the last. "It's something you said you would do, and you do it," said one stoically.)

For scientists, the interesting question is who will go first, and why. A few years ago, they published a simple but powerful analysis of the cohort's decline. They counted up how many medical disorders each person had out of a list of 15, including cardiovascular disease, hypertension, raised cholesterol, diabetes, obesity, osteoporosis, psychiatric problems, cancers and respiratory disease. They found that a whopping 85% of the cohort had at least one of these conditions and that, on average, they had two disorders apiece - even thought most of those people, when asked, said that they were in good health and a large number of the conditions had never been previously diagnosed. It was a sobering picture of ageing, en masses, and one with important implications for Britain as a whole. The cohort study is acting as a bellwether, an early indicator of the tsunami of illness that our rapidly ageing population is going to bring in its wake.

Another recent study showed that people in their 50s who struggled to grip strongly, stand up from a chair, and balance on one leg with their eyes closed had a higher mortality rate over the next 13 years than those who had done well. This suggests that quick and simple tests, carried out in middle age, might identify those people in the population who might benefit the most from intensive medical care or help to improve their fitness and health.

All of this means that the cohort studies are returning to their roots. At the start of its life, the first study shaped maternity services in the fledgling NHS; perhaps now its observations could guide the NHS as it prepares for the onslaught of age-associated disorders that is heading our way.

There is much more: the extraordinary British birth cohort have explored everything from the impacts of pollution, to divorce, to the genes involved in disease - and a roster of dedicated scientists continues to drive them forward today. The cohort studies are still holding a mirror up to Britain - and showing us that we are older - and perhaps a little wiser too.