Brings tears to the eyes... of hysterical laughter, or sadness?

Figure 1. A graph representing the essential calamity that is the human conditionYe gads. A while ago former NIGMS director, Jeremy Berg, posted this dispiriting article which included a graph showing the correlation between the proposal score of funded NIGMS RO1 applications and the impact of the subsequent research conducted by the investigators (Figure 1).Well, with a h/t to Drugmonkey, I was made aware of this new similar study from Danthi et al (2014) looking at the same correlation for the NHLBI. See Figure 2 and weep. I mean, at least the NIGMS one shows a general, if noise-obscured, trend indicating that there is some ("A wee dram, madam! A perceptible sniff!) of a correlation between score and impact, but for the NHLBI? If it wasn't for that slightly bizarre dip around the 15%ile mark (what's with that?) the relationship would be as flat as a Kansas skyline.Figure 2. More weirdness.One can only wonder what the data would look like in a hypothetical universe in which every application submitted was funded. Presumably there must be a threshold at which impact drops to nothing with increased score, but where would it occur? The NHLBI data is no help judging by any extrapolations applied to those figures.The bottom line is that these data do not make for compelling evidence that the peer review process, which is an expensive, labor intensive and time-consuming business, is actually doing a whole lot to identify and support the hot science from the not-so-hot sc...
Source: Across the Bilayer - Category: Medical Scientists Source Type: blogs