Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The bigger issue is what percentage of papers contain enough details to even attempt to reproduce.


In over 80 volumes of ASTM publications, I would estimate they may amount to more kilos than that.

Almost all of the actual lab work requires statistical determination of repeatability & reproducibility to be calculated between different labs, and the summary is included with each document.

I would say there is way less than a kilo without this.

And the amount of supporting raw data on file amounts to kilos which dwarf the pages published. Formally accessible so everything can be thoroughly reviewed at any time in the future, allowing for complete reconsideration if called for.

Scietific instrumentation doesn't stand still.

So it definitely can be done. Even if it's to the extreme not suitable for everybody else.

The less-reproducible documents are there, they did the best they could, but have a smell not shared by the good stuff. You know "exactly" how good or bad the underlying science turned out to be in the real world.

Paradoxically, or intuitively, as the case may be, if you're going to utilize the less-reliable techniques (most likely because they're the best there is), you may need to know how bad they are most of all.

Maybe other publications should raise the bar on statistics as appropriate, I figure zero statistics is about as far as you can get from ASTM "standards".

Some places probably have a lot further to go than others and it would be nice to have a whole lot sweeter smell all around.


Or you could, you know, email the authors for the original data, etc. when planning your reproduction study.


I thought it was understood that's what they do these days.

People can probably appreciate that lot of the kilos are more figurative than ever, but there's still enough hard copies made to fill big trucks though.

I'm not in academics professionally myself, how universal would you estimate it is among their journals these days to require a statistical study between an adequate number of different labs before final publication?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact