Follow the (hidden) Science

For many years, and verified by study after study, sociologists and child development experts have noted that the youngest children raised in wealth develop faster and better academically than their cohort children raised in poverty. Duh, you might think. But the real question is whether it’s the money or something else associated with the wealth/poverty conditions. Perhaps rich parents spend more time with their children (including two parent vs one parent households), or buy/read more books to them, or hire more qualified care-givers, or provide better nutrition and so forth. If it’s just the money, it points to an obvious solution.

In 2018 a group of researchers decided to take on the challenge of studying the issue. It was not just a theoretical assignment. Real debates were going in Washington, DC (and elsewhere) about direct monthly payments to poor parents of young children, along with related proposals for Universal Basic Income (UBI, the idea of a cash supplement available to all people from the government). The researchers developed the Baby’s First Years randomized control trial: the gold standard among scientific research. They identified one-thousand racially and ethnically diverse mothers (from New York, greater Omaha metropolitan area, New Orleans, and Minneapolis/St. Paul) with incomes below the U.S.federal poverty line, whom they recruited from postpartum wards in 2018-19, and randomized to receive either $333/month or$20/month for the first several years of their children’s lives. The $20 group was the control, representing an amount which induced the mothers to participate, but not enough to make a difference in their children’s development. The $333 group may not sound like much either, but it represented an 18% increase in their available income, a sum designed to elicit a positive change. The study was planned to last forty months, but they extended it twice to a total of seventy-six months.

The rigor of the study is unquestioned: children were routinely tested for four primary child outcomes (language, executive function, social/emotional development, and resting high-frequency brain activity) as well as three secondary child outcomes (visual processing/spatial perception, pre-literacy skills, and diagnosis of developmental conditions).

By early 2022, the team released preliminary results: children in the $333 group were more likely to show brain activity patterns associated with the development of thinking and learning. The results hit just as Republicans in Congress had torpedoed a Child Tax Credit, creating a cascade of bad press. Press reporting and politicians skipped mentioning the usual disclaimers: it was only a preliminary result, it was only one of seven possible measure areas, the results were suggestive (even such a well-designed study cannot be definitive, after all). NBC led with “Giving low-income families cash can help babies’ brain activity” and “No-strings-attached subsidies for low-income families improved brain activity in infants, a novel clinical trial finds.” The New York Times headline was “Cash Aid to Poor Mothers Increases Brain Activity in Babies, Study Finds” but then immediately added the political spin “The research could have policy implications as President Biden pushes to revive his proposal to expand the child tax credit.” And other legacy media wrote/led with much the same.

In May of this year, the team publicized their final results. What, you didn’t hear about it? No one did. Here is the final outcome: After the first four years of the intervention, we find no statistically significant impacts of the cash transfers on four preregistered primary outcomes nor on three secondary outcomes. Zero. Nada, Ziltch. The preliminary finding of increased brain activity washed out when all the data was accumulated; it happens.

How did I find out? Yesterday, the New York Times ran with this headline: Study May Undercut Idea That Cash Payments to Poor Families Help Child Development” with the subtitle, “Rigorous new research appears to show that monthly checks intended to help disadvantaged children did little for their well-being, adding a new element to a dispute over expanded government aid.” Kudos to the Times for even uncovering the report, but I do note they re-introduce uncertainty they didn’t show (“May Undercut . . .”) when they liked the preliminary result. And the secondary language (“adding a new element . . .”) fairly runs away from the obvious.

Speaking of running away, the research team quietly completed the study without publicizing the results, just formally submitting them. The Times buried this point in the article, although it did also note that several co-authors declined to comment on their work.

The study results speak for themselves. Several outside experts wonder whether the pandemic somehow skewed the results, but it is unclear how that would happen, or what to do about it. I do add that two additional findings undercut another common argument: the high-cash mothers in the study did not spend the extra money on alcohol and cigarettes, at least according to self-reporting. Also, they were less likely to work full-time, and reported higher stress than the low-cash mothers.

What to take from all this? Unlike the media and political left which ran with the story as a scientific fact when they approved of the preliminary results, I’m not sure it is definitive in its final form. Maybe the pandemic was too large, the stipend was too small, or maybe the kids will improve academically later in life. Maybe. The real lesson here is how science was used as a political weapon. Acclaimed when it confirms one side’s views, literally hidden–by both the researchers themselves and the media–when it does not. There is a related problem in the sciences called the “file drawer effect.” It happens when scientists simply don’t publish negative research findings; they simply drop them in their files to disappear. This has the effect of letting other scientists end up re-creating the same research rather than building on the negative outcomes, so it wastes resources. But it also indicts the scientists’ objectivity, as they put the outcomes they desire above what the data show.

I haven’t seen any other coverage, especially in the legacy media sites which initially reported. Maybe it’s coming. But the next time someone pipes up with “follow the science,” ask them about the Baby’s First Years study. It’s hard to follow what is hidden.

7 thoughts on “Follow the (hidden) Science”

    1. Your comments on what you appear to think is biased or hidden science (I take that to mean liberal views in social science) that tried to hide contrary evidence is mystifying. I simply do understand why you think it was hidden. It was not. The paper was published by the National Bureau of Economic Research, not exactly an unknown organization that has been around for over 100 years and is funded by various foundations. So it was not hidden. Specialists in this area clearly have seen it and have discussed it. That is exactly why a number of them are quoted in the Times.

      As you pointed out, and the research itself notes (the actual paper is at https://www.nber.org/papers/w33844 and can be downloaded as a.pdf), the authors themselves not sure if a number of extraneous factors, including the Covid era, had an effect the children’s development.

      Basically, they are not sure if the study caught important variables or even how definitive it is. Your assumption is that it is because it shows that money does not help poor kids and the authors shied away from this. They did not do so. But the effects of poverty, possibly bad housing, poor neighborhoods (where else would poor people live?), may have had an effect. A number of people who commented in the Times’s article on the research have raised those issues.

      The study also notes that the $333/month payment equaled 18% of the mother’s income (and later only 14%). That means the mothers had an initial income of about $22,200 per year before the $4,000. True, some income was supplemented for a while by other Covid-era payments that eventually stopped. The study says that the mothers also already all had at least 1-2 children before the birth of the research cohort. So, the approximately $4,000 per year that the research study provided helped, but we are talking about a family of at least 3 and possibly 4 (if we include fathers, and only 38% of the mothers were living with the biological fathers). A total of $26,000 per year is not great. And many of the authors (in a separate study I cite below) have noted that whatever payments parents received may have had to be diverted to other family issues or crises (like rent?).

      As the authors and researchers cited in The New York Times noted, the money alone was possibly not enough to make a real difference.

      To make your point that the researchers are trying to hush up their own work, you note that: “I haven’t seen any other coverage, especially in the legacy media sites which initially reported. Maybe it’s coming. But the next time someone pipes up with “follow the science,” ask them about the Baby’s First Years study. It’s hard to follow what is hidden.”

      I would respond that neither you nor I are specialists in this area and that many researchers appear to have read the study; some are already quoted in the Times. For example, in addition ,the study was reproduced in Jama Pediatrics (https://jamanetwork.com/journals/jamapediatrics/fullarticle/2834896) along with an editorial titled “There are No Sacred Deer In Science.” Which positively notes the research study as well as one on home nursing visits for children that show no improvement to children’s health. Clearly that writer agrees with the results of the research study on children and cash payments.

      Six of the study’s authors did a different study providing more money ($1,000/month) to parents (not babies’ first years)–see https://www.nber.org/system/files/working_papers/w34040/w34040.pdf, “THE IMPACT OF UNCONDITIONAL CASH TRANSFERS ON PARENTING AND CHILDREN.” Also, if you plug in the title of the study cited in the Times into Google Scholar, you see that this is a rich topic with lots of research pro and con.

      Bottom line: this is no secret being kept hidden from the public. The fact that legacy media is not publicizing it is partly because this is a topic with lots of opinions and often contradictory research. And this one/one study is not definitive. Plus, there are a lot of other more immediate issues taking place in this country every day.

      1. It’s really very simple. When the preliminary results came out, the researchers involved went to great lengths to publicize it, gave many interviews, and generally promoted the results. When the final results came out, they posted it where they practically had to. As even the Times notes, many of them did not respond to a chance to comment.
        To be charitable, maybe they were disappointed in the results, and felt all their work didn’t amount to much. But, in light of their earlier publicity, they owed it to other researchers and the public to give as much attention to the final results as they did to the preliminary.

        As to the press, your note that “there are lots of opinions and often contradictory research” falls flat, in that it didn’t stop them from trumpeting preliminary results they liked. The media has an even greater responsibility to cover this, and I applaud the Times doing so. Where’s the rest? Still nothing among the other major legacy media.

        I noted in my post that there are significant complicating factors and questioned what–if any–conclusions could be drawn (“I’m not sure it is definitive in its final form. Maybe the pandemic was too large, the stipend was too small, or maybe the kids will improve academically later in life.”). If the difference in the research team’s actions and the press coverage don’t concern you, that’s fine. But you can’t ignore the size of the difference.

        One last comment: you posted the link to the study results in your comment. I already included that as hypertext in my post, so it made me wonder: do you not see hypertext links when I post them? I usually link to all the original sources in my posts.

        1. I find your comments still puzzling. What exactly did the researchers owe “to other researchers and the public to give as much attention to the original results as they did the preliminary”? Owe is an odd term here. They completed a major social science study and published it; it appeared in any number of scientific venues and even as a prepublication in December of last year. They did not keep it a secret. They have a web site.

          If you plug in the study’s title into Google or DuckDuckGo, you will find a prepublication version at Scholars at Duke and ResearchGate; the study was also published in December 2024 by the American Psychological Association and republished in Nature, Europe PMC, the Journal of Public Economies, the National Library of Medicine, etc.

          Also, the same authors published earlier elements of the study over the past few years.
          If you go onto Baby’s first Years Study page ( https://www.babysfirstyears.com/) and go to the News and Media section (which is clearly not up to date, since it does not include the Times story), there is an interesting item from July 2024 from Vox.com (https://www.vox.com/future-perfect/359737/researchers-parenting-mom-stress). The author writes:

          “Usually, when I talk to researchers about new papers of theirs, I have questions for them. When Lisa Gennetian, a professor of public policy at Duke and one of the lead researchers on Baby’s First Years, reached out a few weeks ago to talk about the experiment’s latest findings, she was calling with a question for me.
          The researchers had a slew of interesting findings, she explained, but the picture those findings painted was messy and nuanced. How, Gennetian asked me, do we get that across, especially in an environment where simple messages like “this program worked” or “this program failed” are easier to spread?”

          I should note that the title of the article is honest to the study: “What a big experiment giving money to parents reveals: Giving people cash makes them less poor. It doesn’t fix everything.”

          This partially answers your question. At least one of the authors did reach out to talk about it but understood that this study was not the final word, as I noted in my previous comment, and it was “messy and nuanced.” And again, while this is not the last word on the topic, there are others that diverge from it. I say this because social science research is often less definitive than “hard” science research—there are many variables in human lives that are hard to quantify and separate out and put into neat timelines. You yourself have pointed out these various complicating factors.

          Did the other researchers talk about it? Will they? It may be too soon to say—the final version of the study just came out.

          Given the shrinking state of legacy media, the falling numbers of reporters (just look at the number of reporters who have left the Washing Post in the past year), and the daily chaos that just American scientific research and universities are in and have to be covered, along with a myriad of other domestic and foreign policy issues, this study may not have received the level of inquiry you want—my guess is partly because it satisfies your political views and you would like the negative aspects of the study more widely broadcast, since you seem to find bias here in reporting. In contrast, I find a diminished legacy media. However, as you admitted, it is not the end of the road on this topic.

          As far as the link goes, I wanted to make it more explicit. No agenda there,

          1. The key to understanding my point is the word commensurate. All the things you cite (eg, legacy media, hard vs soft science, nuance) existed for the preliminary results. The researchers went all out publicizing those results, and while the press (as usual) went further, the researchers didn’t rein them in. There are few if any subtleties in the original coverage. Yet when the final results came out, the team just submitted them. I would bet it was the minimum they had to do as a result of whatever funding stream supported them. We’re six months along and the only coverage you cite is in academic journals. I have still only seen the Times coverage in the legacy media. The Times admits it had to dig to get feedback from the team; how come they’re reticent now? How come that singular team member you cite is all about “messy and nuanced?” Where was the language when the team met the press for the preliminary results.
            My suspicion is buoyed by the title from that same interview: “Giving people cash makes them less poor. It doesn’t fix everything.” Balderdash. The study had a specific thesis to prove/disprove, and that wasn’t it. It’s fine to introduce nuance for all the reasons you mentioned, after you report the actual findings. And when you trumpet one set of preliminary findings, you damn well better do so for the final ones.
            Lastly and most importantly, your comment about “satisfying my political views” is terribly telling. You work from that assumption, which is false, to the conclusion I’m being too hard on the study.Perhaps you think I’m suggesting the study proves something for conservatives, a point I made pains to explain could not be the case. I’m actually a huge fan of direct support to families of young children. I was following this hoping they might find the right way to do so, since I think such support could alleviate many of society’s problems down the road. I was hopeful at the preliminary results, saddened by the final ones. But I was disgusted by how the results were handled, both by the team and the media.
            And I didn’t suspect an agenda wrt the links; people read my blog in different forms/different devices, and I’m never sure if things like hypertext show up; thanks for confirming!

  1. 1. TBH, I had never/never heard of this Baby’s First Years study until the Times story, nor had heard any comments on any this from the researchers at any stage. I read 4-5 newspapers/day, but I must be getting too old or distracted. There are news items listed in the media section of the study’s website. But this is not exactly a bonanza in my view.

    2. The study cited in the Times is still a working paper and not the final paper. This implies a way station, not the end.

    3. If you look at the publications page of the study web site, there are some positives to be seen. Just because brain wave activity has not changed is not the only impact of cash grants, which were relatively modest, even at $333/month.. For example, a JAMA article from 2023 (https://www.povertyactionlab.org/blog/2-18-22/strengthening-randomized-evaluations-qualitative-research-babys-first-years-household) summarized some of the work until that point and noted that:

    “The same researchers published 2 other articles that reported on changes in parenting and neurological findings in cases in the Baby’s First Years Study, which align more with the underlying goals of the study. The first of these studies describes how the unconditional high-cash gifts impacted family time and money use during the baby’s first year of life. Families receiving the high-cash gifts increased expenditures on child-oriented items such as toys, books, children’s clothing, and diapers. Furthermore, these expenditures were higher than in the low-cash gift group, and parents also earmarked Baby’s First Year unconditional money for the baby as compared with other government benefits. In semistructured interviews, the mothers described the increased time spent with the baby in reading books, play, talking, and telling stories. The increased time spent on learning was a 10% increase per week, which went a considerable way to match the time highly educated mothers spend in teaching their children. Any concerns in certain political circles that poor mothers would use the cash for cigarettes, drugs, alcohol, and personal luxuries rather than for their children were not found, including by reviewing credit card statements in detail.”

    4. Perhaps you think that this change in poor people’s interactions with their children is not important, but it seems like it could make a difference.

    5. Again, I note that the researchers themselves are not sure why brain activity has not changed or equal what they expected. That and the fact the study is still ongoing may explain their current reticence (as you see it but unseen by me). Given the other positives of their work, they may be trying to figure out what is happening. For an example, see the discussion at the end of the following paper in which the researchers look at the limitations of the money inbthe context of family lives–there are positives they noted, but also constraints (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4940902).

    6. You said that “The study had a specific thesis to prove/disprove, and that wasn’t it.” As I understand it, your argument seems to be that the brain activity did not change much with the extra cash, so this study was ineffective and the researchers did not own up to it.

    7. In response, I note that the study purpose was wider, i.e., to research how extra money affected children and their families, not/not just the last working paper on brain activity cited in the Times. There are both quantitative data and qualitative data. In 2022 the researchers wrote: “BFY aims to answer whether poverty reduction in early childhood (1) improves children’s developmental outcomes and brain functioning, and (2) improves family functioning and better enable parents to support child development” (see https://pmc.ncbi.nlm.nih.gov/articles/PMC8487960/). Yes, brain activity appears to be unaffected but there is more here–this is from their BFY website: “The qualitative study involves four waves of semi-structured interviews with a subset of mothers in two sites over the course of the study to ask about their views and experiences of the cash gifts. These data provide an opportunity to learn about how mothers describe their experiences with the cash gifts in their own words, including the meaning mothers attach to the money, how they decide to spend it, and how they think it has affected their lives.”

    1. As I noted in the original post, there remains some doubt. They keep studying, and maybe years later the extra time with the children makes a difference. But it’s a long shot at this point.

      Here’s the rub: the team had a set of 7 (I think) measures, and they ran out with preliminary results on ONE measure when they were positive. They did interviews, went on TV, you name it. Google for the time period and you’ll see a blitz. Now part of this was the media lapping up results they liked, and inserting them in the public debate during (IIRC) the Biden administration. But let’s be honest: researchers knew what was happening, and were all too happy to play along. Then when all seven measure come back with “no change,” why, they post it and move along.

      Here’s a case study in the same effect, and why it is SO important. Back in 2002, the Women’s Health Initiative did a study of Hormone Replacement Therapy (HRT). Their preliminary results were so striking they cancelled the study and publicized them, leading two generations of doctors to conclude HRT was dangerous for nearly all women. Other studies and some follow on eventually overturned the results, but that took decades, meaning millions of women were treated improperly, and still today you can find medical professionals who haven’t caught up. This is why it’s as important to fess up and publicize all results, not just the ones you like.

Comments are closed.