Weekly reading 16.2018

Witold Kula, Wendy Nelson Espeland, Mitchell L. Stevens, Michael Sauder

Miary i ludzie

In different countries, in reality where non of them has contacted with the other there are the same problems with the metric systems.

Every discipline has its own rankings and should be evaluated by a different metrics. It is almost impossible to have universal measure. Maybe a time span could be an interesting metric, but it is almost impossible to measure it and compare the results. The different types of publication (and every product): a monograph and a paper should be measured differently. Montesque: Is it always good to do everything the same? Maybe sometimes it is important to let some things be different. When citizens are acting within the law does it mean that they use the same law everytime and everywhere? How can two papers be compared if they have the exact number of characters. Are they equal or some is better than the other?

The need of measuring is possible to find everywhere where there is a need of dividing some goods, especially wealth (money).

Does the scientometrics had its beginning in the moment when there was too many scholars? At some point it was impossible to give everyone of them an equal share of the funds. We have to remember that metrics were very often connected to the money and a taxation system.

In the system in which there are many different types of institutions there would be many voices favouring some different metrics. Many groups of interest want to have an influence on metrics, which of them should be used and how, because people always want to use metrics in their favour. Almost always there are some thoughts that the cheating might be a good idea (like for example plagiarism). Every system has it weaknesses and almost immediately there are some cheats or walkarounds. There are always some conflicts about the metrics, especially if there are people who want to use an old metrics or they do not want to know much about the changed metric system. For example a seller wants to sell his product for as much as possible and a buyer wants the opposite. We have to remember a person who wants to sell does not have a good position during the process of negotiation. This person may spend so much resources (time) on preparing the product that sometimes has to accept a lower price. A metric can strongly influence a market, people and their acting. A person or an institution that can provide a metric system could have a similar power. The possibility of being able to compare different metrics may give some control over the market, hence everyone should be able to clearly show the differences between them. Representatives of different classes  or groups can be allies and they may fight together for the fair right.

Metrics are almost always a part of some larger system, we can be sure that there is something broader and wider than just the metrics. Those systems have their own tools and ways of measure different groups of measured subjects. Changing the metrics could cause a domino effect, which means that some different things have to be changed as well. A one cause of the causes of a need of the new metrics could be an unification because of a common market. Sometimes people want some more accurate metrics. Science is able to describe the system and characterise it and its links to the society.

Sometimes one does not have to know the exact meaning of measured dimension. However, the knowledge about many measurements can shed a light on a whole phenomena.

A unified metric system could be a common language of a group of people from different cultures and countries. We might ask a question about an impact factor and large databases (WoS, Scopus) a way of describing a scientific work without the geographical or language borders? This kind of a system will be much harder to introduce. An agreement between people on the common metrics is more important than the metrics themselves.

During major changes we can witness an evaluation of a discourse about a new evaluation criteria. However at some point a law has to be introduced because too long deliberation can freeze it and no let it to be introduced ever. There will always be people who will try to manifest their disapproval of the new metric system (they can send some letters or write the manifestos).

Not every system in history was successful and some of them were cancelled before they had enough time to be common between everyone.

Unification on a small groups does not have to be something good because it can make the differences between the groups much larger.

Evaluators have to be very specific and their rules have to be very clear. Every metric system should be clear and should give an opportunity to check if everything was measured within the rules. A Well written law should be clear and oblique to everyone as well as it should be scalable and compatible. Everyone should use it and act within it. It should be very detailed, therefore not to give many opportunities to be bypassed. Some units could be needed to help with the process of introduction and then check if the law was introduced. This system should have to be created centrally nad introduced by an institution strong enough to prevent local authorities before mixing up with a new law. A stratification and a hierarchy have to be well planned and organised. We have to remember that people need some period of time to get used to a new system. A good idea is to do a research before the new law is prepared, it can give some advices what could go wrong and where an emphasis should be put.

One metric is very often idealised, people might see it as an ultimate remedy for every problem.

From the perspective of a person in charge one law system and metrics might be a way of creating a society without differences in needs or dreams. Something a little bit like a dystopia.

Very often we can observe a chain of dependancy, managers on the different levels have their own rules and in many cases the their rules might not match in a 100% the law. They may demand more or something slightly different.

Some possible causes of conflicts:

  • between a country and the scholars;
  • between a people from one “village”;
  • between a person and a boss.

A sociology of quantification

Wendy Nelson Espeland, Mitchell L. Stevens

DOI: 10.1017/S0003975609000150

The note

Numbers sometimes help to direct attention or put an emphasis on an aspect of something.

A speech or a text may be considered as something without a deep meaning, just a statement. However it could be an order or advice or even an act of persuasion.

Quantification may have multiple purposes and meanings. Analysing particular instances of quantification in context can reveal them. Some measures used once to describe something can be used to influence it somehow. Counting might look like something simple and easy, however it depends on a scale. When someone wanted to create a system of quantification of something for the whole country would have to face an issue of having an experienced team of people who would be able to solve many obstacles. Especially if the results could be used for some political or official reasons.

The measures can change the way of how the funds are distributed to schools, universities or even countries.

Every measurement and comparison might cause changes in measured institutions, group of people or how one acts. Every ranking may be a cause of a changing behaviour. Statistics an shape or even create a reality.

It is highly important the measures should be reproducible.

Commensuration creates a specific type of relation among object. It can transform a difference into a quantity.

Numbers help to communicate across the geographical boarders and travel long distances.

The information in many institutions is brought from the bottom to the top of the decision-making ladder. During this transition the information receive a different – more robust form. The nearer the top the numbers become more authoritative and inherit handlers authority.

Aesthetics of the graphic and numerical images is important and it is up-to-date for a limited period of time. It may help to put an emphasis on most intriguing inquiries.

Parsimony (understood as less is more) and clarity are the most valued features of the graphical representation of data.


Rankings and Reactivity: How Public Measures Recreate Social Worlds

Wendy Nelson Espeland, Michael Sauder

DOI: 10.1086/517897

The note

Many measures like performance measures, assessment of universities, rankings of schools could be characterised as an “audit explosion”. Everything now has to be transparent and accountable, because we can observe and audit it.

Reflexivity is a known phenomena which states for a process of changing measured communities, people or social groups. Sometimes it is seen as an error or a methodological problem.

Reactivity is an inevitable result of human reflexivity.

Using quantification could be a way of mitigation conflicts, overcoming distrust and coordination across both social and physical distance.

Reactivity might be a promise or a threat. Self-fulfilling prophecies could be some processes by which reaction to social measures confirm the expected in measures.

Rankings create expectations and people change their behaviour accordingly.