There is an emerging problem within the scientific community – its rapid growth together with advancement of technology increases information flow to unbearable levels. In these conditions existing system for research evaluation is becoming more and more obsolete, creating a bottleneck in the “knowledge production pipeline”, which results in the loss of tremendous amount of data. To illustrate this point “knowledge production” can be formalized as following:

1) Science is a Machine operating the “knowledge production pipeline”. It extracts, filters and refines the information from the Universe.
2) The end product of the pipeline is structured Knowledge ready to be “consumed” by society, industry or by the Science Machine itself (knowledge reuse creates positive feedback).
3) Pipeline consists of four operational segments, represented by the corresponding sets of tools: extraction of information, documentation, evaluation and knowledge dissemination.

The main bottleneck is created in the segment of research evaluation. Scientists (operators of the pipeline) are still performing this process manually. Increased automation of the preceding “extraction machinery” and the shift to electronic “documentation machinery” results in disproportion between the amount of operators and amount of information to be handled. In other words, while we are perfecting tools for knowledge extraction (thus increasing the information flow), the machinery required for knowledge evaluation is worn out, loosing ability to handle this flow.

Another problem refers to inefficiency of the existing documentation tools – paper laboratory journals and local electronic data storages provide very limited access. In fact, this problem also points to the obsoleteness of the science evaluation practices: since no credit is given for the knowledge bypassing the peer-reviewed publishing process, scientists lack incentive to make negative results and «raw» data publicly available.

So essentially everything boils down to the necessity of delivering new metrics for research activities and partially automating the evaluation process. This does not mean elimination of the peer-review, but increase in its efficiency.

And, beyond everything else, academia has to start giving credit to researchers performing peer-review and make its results publicly available, thereby encouraging this type of scientific activity and removing bias and injustice from the process.

///

More reading:

• Mechanisms for citation are the key to changing the academic credit culture
• Peer review: the myth of the noble scientist
• Giving credit, filtering, and blogs versus traditional research papers
• Rewards, output and academia
• Experimenting with Peer-Review

In perspective of the topic discussed here:

The Real Science Crisis: Bleak Prospects for Young Researchers
Tight budgets, scarce jobs, and stalled reforms push students away from scientific careers.

It might sound presumptuous, but in my opinion policy makers are looking for solutions in the wrong places.

Departments and students must recognize that the majority of science doctorate recipients no longer become professors, and that realization should cause a shift in the culture and practice of graduate education. “There’s a mismatch between the opportunities available to students as they complete their work and their expectations and the nature of their training along the way.”

Indeed, as a rule, tenured professor is the only stable academic position, and amount of those is strictly limited. But instead of blaming the educational system for producing too many PhDs and inventing programs to prepare graduates for the job market outside of academia, one could simply devise a new type of a “tenure” academic position. A stable intermediate between a PostDoc and a Professor – with small budgets and limited responsibility, but stable! This would secure a person who is willing to stay in academia, but did not acquire enough luck to reach Professor level. In modern world of rapidly evolving methodology and extensive interdisciplinary research, those people will be of much higher value than one might expect. In fact, given higher efficiency of knowledge dissemination within small research groups, this could also resolve the problem with quality of postgraduate education.

The second problem is harder to spot and uneasy to accept. The source of it lies within the existing “peer-based” evaluation system for science and research, which forms the core of publishing and funding mechanisms. Quotation marks show up for the reason of “peer-based” in many cases functioning in celebrity-based and celebrity-biased manner.
This evaluation system worked fairly well with limited number of scientists, when global awareness of ongoing research was possible in most of the scientific disciplines. However, once electronic media accelerated the rate of scientific discovery, diversifying research world and increasing information flow to unbearable levels, the existing “peer-based” methods became obsolete and destructive. And while policy makers are trying to solve problems by scaling up the funding, the system cracks:

…Congress doubled the NIH budget from $13.6-billion to $27.3-billion between 1998 and 2003. Since then, the agency’s appropriations have not kept pace with inflation, which has eroded the actual amount available for research.

Beyond inflation, immense amount of those resources vanishes in the redundancy of our efforts. Existing evaluation system discourages sharing of knowledge and expertise, leading to unhealthy competition, secrecy, data falsification, and as a result – redundancy of research. One really has to restructure the whole system and ideology behind it, focusing on elimination of pyramidal hierarchy, giving more power to young scientists, introducing new methods for efficient knowledge dissemination and encouraging sharing of research artifacts. This restructuring in turn will release a lot of resources, which can be rerouted back to increase the amount of stable research positions available within academy.

///

There is also a bit of discussion on this topic going at friendfeed.

“Monday Begins on Saturday” is the name of the Russian (Soviet) Sci-fi novel written in 1964 by brothers Arkady and Boris Strugatsky (the “fathers” of Russian Sci-fi). It is a very light, humorous and intelligent story portraying the life of truly inspired scientists, devoted to their research in such a way that their Mondays start (metaphorically) on Saturdays.

It is also a bit of a fantasy style, since the action takes place at the Scientific Research Institute of Sorcery and Magic (SRISM), where researchers are pursuing science with respect to magic objects, creatures from fairy tales and other elements of the supernatural.

“Monday…” could be somewhat more appealing to [bio]informaticians - although the story is not about computers, but the main hero is a geek, possessing rare (at the time) skill of operating one of the first punch-card computers.

A number of people, including myself, would say this novel is one of the most original and enchanting Sci-fi stories ever written. I would even vote to include it into official scientific education programs, at least for PhDs =). Of course human perceptions do differ, but if you happened to be a scientist - I bet you’ll love it!

Actually, the point is that the spirit of the people who are now advancing the concept of Open Science is very much alike the spirit of the SRISM scientists. Reflected in the title, one of the ideas behind this blog is to follow the advancement of the New (Open) Science…

In attempt to generalize the key concepts of Web 2.0 trend… Everything sums up to “information managed by communities”.. Things like service-oriented architecture, grids, clouds, etc. remain within the “web 2.0 tools” entity. Hope analogy with the prism is not too odd…

Constructive critics are welcomed!

so, i have to write 300 characters..science 2.0 , web 2.0, peer-review, publishing, open access, open science, data sharing, social networks, science communities, research, labjournal, future of science, open repositories, common licensing, trust, collaboration, research projects, semantic web, semantic data. zemantification rocks!! =)