Comments on: The next revolution in Science: Open Access will open new ways to measure scientific output https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/ Sharing the results of publicly funded research Sun, 08 May 2016 10:20:40 +0000 hourly 1 https://wordpress.org/?v=4.9.8 By: Luke Angel https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-389 Tue, 10 Jul 2012 05:21:41 +0000 https://access.okfn.org/?p=402#comment-389 This was very helpful information from your side and i like to tell you that please keep posting such an interesting posts like that. I am really waiting for your future posts.
Thanks and regards,
Pharmaexpressrx

]]>
By: Mendeley will have an impact on the library. Workshop presentation: Mendeley Institutional Edition « @keitabando https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-364 Fri, 15 Jun 2012 03:55:13 +0000 https://access.okfn.org/?p=402#comment-364 […] この Altmetrics、オープンアクセス分野はもとより、科学界全般で俄然注目を集めている様な気がしてなりません。(これとかこれとかこれとかこれとかこれとか) […]

]]>
By: Robin P Clarke https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-327 Mon, 23 Apr 2012 21:11:31 +0000 https://access.okfn.org/?p=402#comment-327 Gunter, the WebCite website you mention (haha it sounds same as website in english!) looks a great thing but I don’t follow your sequence of “first publish …, archive it, assign a DOI… So far so good but then how does it get peer-reviewed if it isn’t sent to a journal?

]]>
By: Gunther Eysenbach https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-326 Mon, 23 Apr 2012 13:53:05 +0000 https://access.okfn.org/?p=402#comment-326 Typo up there: “third parties” instead of “throw parties” (damn spellchecker)

]]>
By: Gunther Eysenbach https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-325 Mon, 23 Apr 2012 13:51:27 +0000 https://access.okfn.org/?p=402#comment-325 A few important omissions to note:

1) JMIR has been collecting and measuring social media resonance on Twitter for over 3 years now – we call this the Twimpact Factor (see “Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact” http://www.jmir.org/2011/4/e123/). The TWIF and related metrics (TWINDEX) are displayed here: http://www.jmir.org/stats/mostTweeted

2) altmetrics is really just a fancy term for infometrics, webometrics, social media metrics, infodemiology metrics etc. If you google these terms, you will note that these ideas are not as new as you might think

3) WebCite (http://www.webcitation.org) was also originally set up not only to archive cited work on webpages, but also to collect metrics for impact based on citations/mentionings of webpages (open access articles?). It also creates a “publishing a-la-carte” system (Priem), where researchers can first publish something on the Internet, archive it (or have it archived by throw parties who want to cite it – i.e. create a WebCite snapshot), assign a DOI, have it peer-reviewed, and then view alternative metrics. Perhaps now is the time to fund disruptive systems like this which turn the traditional publishing system on its head?

]]>
By: Robin P Clarke https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-324 Mon, 23 Apr 2012 13:20:12 +0000 https://access.okfn.org/?p=402#comment-324 Indeed those changes would improve the situation but it looks like you are still confusing the level of attention a documents gets with its level of merit. The problem is that (as Einstein supposedly said!) what you can count doesn’t necessarily count and what really counts you can’t necessarily count. The judgements of what is real quality and importance can only with difficulty be formulated into some operational critieria, if at all. I find the academic world, and the world outside academia too, to be grossy overawed with superficiality, and not least infested with the Matthew effect – whereby a statement receives great publicity and credit if it comes from an already prestigious celeb yet the identical statement gets not even mentioned if some non-entity states it years earlier than the celeb. The whole system of trying to assign bulk authority labels of merit ultimately backfires, as per the Lysenkoist catastrophe in which the genuine experts had the status of slaves in Siberian mines while charlatans held all the professorships. There’s plenty evidence that the ghost of Lysenko is still alive and prospering in the Western “free” world.

I would like to see a new system in which the crude measures underlying publish-or-perish are binned (as a random system couldnt be worse) and people just publish everything they think worthwhile directly to the web and then things get a reputation directly from how people react to them. At the moment the key problems appear to be (1) peer review which in medicine at least is well-known to just bias in favour of establishment drivel or mutual backscratching cliques, and (2) pubmed inclusion (and google status) which again bias against anything new and deter people from just putting on web (because of need to be prominently indexed) – and researchers be negatively-treated for their drivel publications such that there is a much lower volume of output and a concentration on only publishing genuinely worthwhile material. Of course all this may be different outside the medical field.

]]>
By: Tom Olijhoek https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-323 Mon, 23 Apr 2012 12:08:31 +0000 https://access.okfn.org/?p=402#comment-323 The point of my article was to show that once you and everybody else are at least able to take note of a complete article , this opens up the possibility of better judgment. I have never argued anywhere that this judgment would be based on fashionableness or newsworthiness. You yourself say ” There is no substitute for actually reading ” and that is exactly the point I am also making.
I agree that he mere citing of scientific articles by peers on the other hand is no garantee for scientific excellency. That is what the impact factor measures, and I would argue that if very many people download an article, tweet about it, comment on it, blog about it , then the article is bound to have significance. Mind you, it is no proof of significance and certainly not proof of a breakthrough. But not many articles are breakthroughs. For me when many collegue scientists and also great number of other interested persons talk and write about a particular paper I would like to take part in that discussion on the basis of having read the open access article. Altmetrics weighs the opinion of other people as a function of their numbers and their activities, not only as a function of the quality of (some) individuals. To me this adds value to the flawed system based on citation assessment

]]>
By: Robin P Clarke https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-322 Mon, 23 Apr 2012 09:31:26 +0000 https://access.okfn.org/?p=402#comment-322 This comment is not to praise the above article but to point out some of its serious weaknesses. And yet I guess some quasi-brilliant quality-assessment metric will count this comment as proof of the article’s superiority rather than inferiority.

The author seems to be entirely lacking the concept that popularity/fashionableness/newsworthiness has only marginal or even non-relationship with real quality. Indeed can well be inversely related. For instance Wegener’s continental drift ignored by professionals for 50 years, Mendel’s genetics experiments ignored for decades, Dr Down’s descriptions of what are now called late and early autism, not even registered by the majority of autism researchers even now, the list goes on. The greatest discoveries have been notoriously COMPLETELY IGNORED (Impact Factor zero, H factor zero) by the “peer” community rather than salivated over with whopping citation impacts.

The same nonsense with so-called peer review which has been repeatedly shown to be nothing to do with quality but only with conventional wisdom fashionableness and hence again strongly negative against the real breakthrough discoveries.

There is no substitute for actually reading and evaluating the thing itself with reference to the actual evidence. The real sci lit problem is the publish-perish system which is drowning the genuine science under huge oceans of drivel. For this reason a high proportion of professional science contributes negatively rather than positively, not that any sorts of impact factors could ever tell you so..

]]>
By: Jan Velterop https://access.okfn.org/2012/04/19/the-next-revolution-in-science-open-access-will-open-new-ways-to-measure-scientific-output/#comment-306 Thu, 19 Apr 2012 09:23:59 +0000 https://access.okfn.org/?p=402#comment-306 Thank you for mentioning Utopia Documents. Its speed depends on your own bandwidth and the speed with which the services Utopia Documents links to respond. You mention that it may be a disadvantage of UtopiaDocs that it focuses on the PDF format. But that is precisely the point, to focus on PDF, where the deficiency is in terms of web connectivity, while a lot of scientific literature is still ‘ingested’ via PDFs. Links are widely available with HTML versions (though not as widely as one might wish), but bridging the connectivity gap between HTML and PDF is one of Utopia Documents’ main goals. In addition to Altmetrics, Utopia Documents plans to include other metrics as well in a forthcoming release.

]]>