In search of a Grand Unified (sciento-metric) Theory of Everything…

“Whether you can observe a thing or not depends on the theory which you use. It is the theory which decides what can be observed” — Albert Einstein, objecting to the placing of observables at the heart of the new quantum mechanics, during Heisenberg’s 1926 lecture at Berlin.

“I believe in intuition and inspiration. Imagination is more important than knowledge. For knowledge is limited, whereas imagination embraces the entire world, stimulating progress, giving birth to evolution. It is, strictly speaking, a real factor in scientific research” — Albert Einstein, Cosmic Religion : With Other Opinions and Aphorisms (1931).

“It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience” — Albert Einstein, Philosophy of Science, Vol. 1, No. 2 (April 1934), pp. 163-169., p. 165.

“If I would be a young man again and had to decide how to make my living, I would not try to become a scientist or scholar or teacher. I would rather choose to be a plumber or a peddler in the hope to find that modest degree of independence still available under present circumstance — Albert Einstein, letter to the editor of The Reporter (13 October 1954).

Just as the Relativity Theory inspired some Cubist painters to include the effects of space and time in their work, the search for a Grand Unified Theory of everything in theoretical physics has spilled over into socio-economics. Ratings, rankings, Impact Factors and Hirsch Indices galore are abound in nearly every sphere of evaluation of the individual. But, policy-makers, research-staff hirers and journal editors have all fallen into exactly the trap that Einstein warned us of in 1934. Over-simplification. Ease of application of sciento- and biblio-metrics without regard for, as Einstein said, “intuition and inspiration” (read as “quality”) means that the Grand Unified Scientometric Theory (GUST) of everything is hot air. It leads to decisions based on what it can observe rather than observing what there actually is.

In Einstein’s General Theory of Relativity, the understanding of the spacetime continuum comes from an understanding of the topology of mass. Its shape (defined by the spacetime-metric) leads to the emergence of the thing we call Gravity. Just as mass (the topology of the spacetime-metric) is the source of gravity in Relativity, quality “measured” by the biblio-metric is the source of Citation Impact. A correct description of topology leads to correct values of the spacetime-metric and a quantification of Gravity. A correct description of quality leads to correct values of the biblio-metric and a true quantification of Scientific Worth. In Scientometry, quality is the analogy of topology. But Scientific Worth and Citation Impact are not necessarily the same thing.

So why all the fuss and hoo-hah? Now we can understand. The biblio-metric is wrong. While quality should determine the values of the biblio-metric, it doesn’t because the plethora of blbio-metrics being used do not depend on quality. An ojective way of measuring it hasn’t been found yet. Instead, we construct incorrect biblio-metrics, distorted by the adoption of incorrect proxies for quality like the Impact Factor or the Hirsch Index. These then lead to perceived citation impacts for the individual scientist who, as Einstein said, probably wish they had become plumbers.

While I applaud the attempt the effort last month by Nature to discuss scientometry in its special issue Science Metrics, without a quantification of quality, it is all just a GUST of hot air. Scientists and bibliometricians, serious about the issue should begin immediately looking at ways to measure quality. The Free Science Flying Circus tried 5 years ago using neuro-fuzzy neural networks and are still trying:

• Perakakis P, Taylor M, Buela-Casal G (2005) A neuro-fuzzy system to calculate a journal internationality index. Proc CEDI2005, Symp. Fuzzy Logic and Soft Computing, CEDI, 157-163. (3 citations)

Until then, we strongly urge evaluators to refrain from the application of biblio-metrics without understanding the quality of each individual’s submitted work. These measures just dont and wont add up. C.V.s littered with self-citations and inflated impact factors & h-indices due to high citation traffic from Letters to the Editor or Editorial Comments or due to provocations resulting from the publishing of speculative, wrong or unjustified results, are just some of the symptomns. It’s time we go back to basics and look at quality. Quality science creates quality scientists. Bibliometrics has work to do.

Advertisement