Golden Fleece 2.0?

Note: Recently, several stories - here and here, for example - have reported the political attacks on the National Science Foundation and its peer review system. The assault seems especially to target research in the social sciences that the agency sponsors. I asked Melinda Baldwin - who is working on a history of peer review (and whose fascinating new book on the history of the journal Nature will appear next year) - to contribute a guest post. Lindy's thoughtful and timely contribution is below... Senator William Proxmire made a career out of taking principled and stubborn stances. Proxmire, a Wisconsin Democrat who held office from 1957–1988, was an early critic of the Vietnam War, an early advocate of campaign finance reform, and a jogging fan who authored You Can Do It!: Senator Proxmire's Exercise, Diet and Relaxation Plan.Proxmire is best remembered, however, for his Golden Fleece Awards, a monthly badge of dishonor he bestowed upon the federal project that he deemed the month’s worst use of taxpayer money.While Proxmire’s “prize” eventually graced almost every federal agency, one of his favorite targets was the National Science Foundation—an organization that, Proxmire believed, frequently funded frivolous projects that were of no benefit to the American people.Fast forward four decades -- Proxmire seems to have found a successor in Representative Lamar Smith (R-Texas), the current chair of the House Committee on Science, Space, and Technology. Last summer Smith made headlines when he requested peer review reports from five National Science Foundation grants that he felt were questionable.Smith has also followed in Proxmire’s footsteps by publicly mocking individual grants. And last month, ScienceInsider reported that Smith has successfully obtained access to the referee reports on fifty NSF grants as part of an investigation into the NSF’s inner workings. Smith told ScienceInsider the review will continue “until NSF agrees to only award grants that are in the national interest.”Many scientists and policymakers have worried that Smith’s investigations will damage the NSF’s peer review process. Representative Eddie Bernice Johnson (D-Texas) called Smith’s actions a “campaign against NSF’s merit-review system,” and the NSF has protested that reviewers submitted their reports under a promise of confidentiality.With Smith’s investigation still ongoing, it is worth considering the history of peer review at the NSF—and examining how the clash between the NSF and Proxmire played out nearly forty years ago.It was not until after the Second World War that external refereeing came to be seen as an essential feature of a respectable scientific journal or grant-giving organization. ((Most observers trace peer review’s history back to Henry Oldenburg, the Secretary of the Royal Society and the man responsible for managing the Philosophical Transactions of the Royal Society in the seventeenth century. This story, however, is a bit too simplistic. Oldenburg did solicit opinions on papers that he was considering for publication, but far more casually than the term “peer review” implies. The first formalized refereeing procedures emerged at scientific societies in the 18th century. The use of external referees spread slowly and haphazardly over the course of the nineteenth and twentieth century, and many organizations continued to place decision-making power in the hands of editors or directors well into the twentieth century.)) After the National Science Foundation Act of 1950 established the NSF, the new Foundation’s leadership quickly decided that they should consult outside experts for opinions on proposals. Some proposals were sent out for “ad hoc” mail review; copies of the proposal would be mailed to scientists, who would then submit their comments by return mail. Others were evaluated by panels of experts assembled in Washington to read proposals and prepare reports on-site. The choice of the reviewers, as well as the choice of panel review versus mail review, was left up to NSF employees.In 1975, the NSF’s peer review process became the center of a major public controversy when Proxmire bestowed the first two Golden Fleece Awards on NSF projects. The first “prize” went to a sociological study at the University of Wisconsin about interpersonal attraction. Proxmire declared that the NSF should “leave some things in life a mystery” and “get out of the love racket.” A second award went to psychologist Ronald Hutchinson’s study of why humans, rats, and monkeys clench their jaws in moments of stress. Proxmire called the study “nonsense” and complained, “The good doctor has made a fortune from his monkeys and in the process made a monkey out of the American taxpayer.” Proxmire’s lively and cutting press releases won widespread media attention—and led Hutchinson to sue him for libel. ((Proxmire eventually had to make a public apology, and his opponents widely ridiculed him when it was revealed that his legal defense had cost over $100,000 of taxpayer money.))But Proxmire's criticisms also struck a deeper nerve. The American economy was mired in stagflation and still reeling from the recent oil crisis. Media attention placed Congress's attention squarely on the NSF's spending. Congressman Robert Bauman even proposed an amendment that would have required the NSF to obtain Congressional approval for each grant. The controversy culminated in six days of oversight hearings about the NSF's peer review process in July 1975. Nearly everyone at the hearings agreed that peer review was the best method for deciding which grants should receive funding. Congressman John Conlan even argued that the NSF wasn't relying enough on peer review and instead put too much responsibility in the hands of NSF staff. ((Conlan  and Bauman focused their ire on one NSF project in particular: a controversial social sciences curriculum called “Man, A Course of Study” (MACOS).”))The outlier was Proxmire, who argued that the concept of peer review was fundamentally flawed. In a written statement, Proxmire declared, “I have received a number of letters pointing out that the peer review system serves to perpetuate the funding of established people, ideas and institutions.”Proxmire seemed to hold out little hope that peer review could be anything other than “incestuous.” He argued that NSF reviewers would favor proposals whose research had been published in peer-reviewed journals, journals would look favorably on articles that had received peer-reviewed NSF grants, and that “we come full circle, when we realize that many of the top researchers in a scientific field have been recipients of NSF grants, reviewers of NSF grants and finally editors of their technical journals.”Proxmire’s criticisms found little support at the oversight hearings, however. Subcommittee Chair James Symington (D-Missouri) declared that “witnesses overwhelmingly agree” that the concept of peer review was “fundamentally sound” and should be used at the NSF. Meanwhile, the NSF leadership responded to Conlan and Bauman’s criticisms by creating a new audit office that ensured the NSF was placing appropriate emphasis on both positive and negative peer review reports. The NSF also began providing verbatim copies of referee reports to proposers—a practice that continues to this day.How does the 1975 controversy echo in the 2014 remix? Unlike Proxmire, Rep. Lamar Smith has insisted that he does not wish to alter the NSF’s peer review process. But Smith’s criticisms of individual grants and the demand for the NSF’s referee reports strongly suggest that Smith does not believe that reviewers are making good recommendations about which projects are in the national interest. In 1975, Congress and the NSF placed trust in peer review to ensure that NSF grants would support both excellent basic research and national interests. In 2014, Smith seems to believe that this trust was misplaced.From a historian's perspective, what remains fascinating is that attacks on NSF grants have come from both Democrats and Republicans. Beating up on the NSF is a bipartisan pastime. More seriously, Smith's current crusade raises a real policy concern for the scientific community: if the House Committee concludes that the NSF merit review process is not yielding desirable outcomes, as seems likely, what sort of review might be a suitable replacement? Will Rep. Smith, like his predecessors, suggest that Congress should have veto power over individual NSF grants? The irony here is that, just as other countries like China are building research ecosystems based on the U.S. peer review model, some politicians in our country seem intent on undermining the one we already have.