discussion of Mihaljević et al. 2019
The article “Reflections on Gender Analyses of Bibliographic Corpora” by Mihaljević et al. (2019) discusses the problems of a bibliometric analysis of research publications with respect to gender mainly in the field of Science, Technology Engineering and Mathematics. The authors notice the lack of data that currently prevents a decent analysis and request a development of a sustainable solution. In the end the only feasible approach appears to be an open science approach as it can also cover the additionally discussed methodological problems of dealing with the findings of a quantitative analyses of data. Besides this, as their “ultimate goal” the authors depict “to foster a discussion (…) when making statements about individuals and existing societal issues based on publication metadata.” (Mihaljević et al., 1.) Okay, here you go 🙂
In the first place, the authors motivate a high interest in the connection between gender and the under-representation of women in academic positions and publications. They even call it “a natural question” that asks for the contributions of different publication practices of social groups to the observed gender gap (Mihaljević et al., 1.) On the other hand, they state that it is basically not possible to perform a decent analysis due to a lack of personal information in the data sets (Mihaljević et al., 2.1). They also raise methodological concerns in generating these data and point on the danger of misinterpretation of the derived findings leading to an enhancement of inequality when they are taken without qualitative contextualization (Mihaljević et al., 2.3). An earlier finding of the authors is taken as an example, that show that women publish in less prestigious journals (Mihaljević-Brand et al. 2016). This observation could be taken as an argument to rectify why women are less represented in academic positions. Those rush conclusions can be avoided by a careful evaluation of the study and its conditions. To put the challenge in a nutshell: The metadata of publications are agnostic to social aspects as gender. The science system should, but is not.
It is true: the default bibliometric data set, such as MARC21 or as RDA, do not register the gender of authors of a publication. They give a name string that often enough is only one derivate of the name of an individual, due to marriage, multiple first names or transliteration, especially if different alphabets of the original name had been involved (Mihaljević et al., 2.2).
Actually, the gender-blindness (among other blindnesses) of publication metadata is not only a disadvantage: For domain specific investigations, the absence of personal details can be considered as nothing less than a condition for research. Individual information could even corrupt an unbiased view on disciplinary topics. For meta-analyses as we do in bibliometrics, instead, these details would highly improve the understanding of the mechanisms of science as a social field (Seidlmayer et al. 2019). Bibliometric analyses are an important technique for the investigation in the development of science (Gingras 2014, 18).
This distinction between domain specific research questions and the bibliometric meta-questions, might not be taken sufficiently into account by Mihaljević et al. when they generally refuse statements on sensitive topics and “keep questioning the necessity of any given gender-related data analysis” (Mihaljević et. al., 3.). In the context of the given huge social gaps already mentioned, the reaction to the methodological problems should rather claim for a better data basis and a clearly defined usage of the outcomes than to stop the investigation. Data analysis on social circumstances can provide us stable arguments in favor of the marginalized groups of people. In order to ensure a positive outcome the data basis needs to be improved along with the critical handling of the results.
First, we need to adhere something else: If we want to reach a more detailed picture of social privileges and disadvantages we need to broaden the scope of social aspects from gender to many more that lead to disadvantages in academic careers and publication behavior. Besides gender, other social characteristics, such as the social background of the family, the country of origin or the financial power of the institution affecting equipment and paying of publication charges highly influence the research and publication behavior.
Even further go the history of sciences and sociology of sciences, from Ludwik Fleck to Bruno Latour, who discuss the laboratory and the processes within as social shaped – starting with the relations between individuals to the gestures during experiments.
Therefore, a strong interest in a better understanding of the social aspects of science would be appreciated. It is not only gender – although it is considered often as a strong identity aspect, that needs to be taken into consideration to gain a more extensive idea of the complex issue. I guess, also Mihaljević et al. will agree to this broader approach. It comes with similar problems.
However, the main issue for the bibliometric research addressed to publications is a matter of data. The main problem to face is to obtain them. Unsatisfactory are techniques as automated name-allocation of the name-string applying algorithms as Mihaljević conclude convincingly (Mihaljević et al., 2.1) that is offered for example by genderize.io platform. Such approaches of the depiction of gender are justifiably refused as they are a third-party declaration of a very individual characteristic as the gender role. Such techniques can be rejected even more as they perform often in a binary manner. Transgender, inter or other persons will necessarily be misclassified in such heteronormative approaches (Mihaljević et al., 2.1).
The only really convincing method to gain information on issues as gender appears to include the people themselves that are the object of the investigation and ask them on their self-declaration. This comes to the cost of a high effort in time and organization. Furthermore, it needs to be done again and again for every investigation (Mihaljević et al., 2.1).
Wikidata and ORCID as a solution for the missing data?
In the first place, I thought about ORCID and was sure it would solve this. The core concept of ORCID is the self-description that avoids mistakes by third parties assignment of publications due to change of name or other. I was sure that ORCID would be the perfect place for self-declaration and could be used for bibliometric analysis filling up the required missing information. Yet, this is not the case. ORCID does not record such kind of personal information.
Why ORCID deny to register self declarations in the case of gender can be read in the “ideas archive”. Here an interesting conversation on the thread “include gender“ is stored and declared as “not planned”: User Joe Richmond asks to implement gender field since it would help him to contact researchers: “With Chinese, it is often impossible to know and then women are being contacted as if they were men.” Some years later a second user. Lee Raye, a Tutor Swansea University, broadens this argument when she/he/they adds: “In academia we regularly have to talk about people in the third person “Her research suggests that…”. It is not always possible to know someone’s preferred pronoun (e.g. “he”, “she”, “they”). This is not just if the researcher has a gender-neutral name, but also if they are transgender. Having the gender or, especially, a pronoun listed would be very useful.” As the demands aim for a personal contact or an imaginary discussion in an academic text, they are differently motivated than our concern of a bibliographic meta-analysis that looks at the large scale but still convincing matters.
The ORCID-support replied as follows: “[…] ORCID records are specifically designed not to collection personally identifiable information (other than email address). The information on an ORCID record is about the career and research of an individual; ORCID is about linking the researcher to their research-outputs, not providing personal details.”
I understand, that when focusing on the domain specific discussions social aspects should not play a significant role, as we stated above. Although, ORCID might consider not to register academic positions and funding as well if they truly don’t want to publish personal data. In our problem of the missing data for bibliometric studies, ORCID cannot help, at least in the case of gender. But, maybe with information on the career.
At least, the problem of binary gender declaration can be solved easily, as far as I see. Platforms as Wikidata allows the widespread of gender descriptions in as attributes.
In the case of gender for humans, the following Wikidata provides the following items: male (Q6581097), female (Q6581072), intersex (Q1097630), transgender female (Q1052281), transgender male (Q2449503). Those items can still be wrongly related to a person by a third party. The second critic of Mihaljević et al., the self-declaration of people, remains an issue. Most data items on authors in Wikidata are not created by the persons themselves, though, they can be supervised and edited by them.
Therefore, Wikidata might not represent the perfect database for our concern. Though, it is the best we have (as far as I see), if we do not want to cease bibliographic analyses on sensitive social matters.
Quantitative approaches always need a critical inspection
Besides the trouble with the provided data, Mihaljević et al. emphasize a methodological problem concerning the outcome of the data analysis that maps individuals to statistical classes and points on the strong relation to its specific context.
While knowledge relying on an unsatisfactory data basis can only be “approximate knowledge” (Tukey 1962, according to Mihaljević et al., 2.3), Mihaljević et al. go even further and apply this critic to the quantitative analyses itself. Referring to Angermuller and van Leeuwen (2019) they claim, that any quantitative analysis that tries to map is always a factor to the social world it aims to describe (Mihaljević et al., 2.3). They affirm it always reproduces the given circumstances and argue, taken with Foucault, bibliometric research is a mechanism of power.
However, should this critic based on the shoulders of Foucault not pertain to every description and research output? Every interpretation is a single view on a situation. It can always be only a detail and has to be undertaken a critical inspection that includes a reflection on its context. Almost every finding has to be understood as “approximate knowledge” and needs to be discussed related to its conditional statements.
Mihajević et al. claim for reproducibility of research but also emphasize the effects of social characteristics influencing the research. At this point their argument of non-declaration of social characteristics introduced above (Mihaljević et al., 3.) reasoned by its possible transformation into a tool of power (following Foucault) turns to its opposite: For how could we reproduce the research if many influencing aspects – as gender et al. – are not made transparent? The demand for non-declaration rather prevents a reproducibility and does not defend the individual against the statistical arguments.
Does the demand for FAIR principles (findability, accessibility, interoperability, reusebility) not include a self-declaration, as it is common practice in some social science (Wilkinson et al. 2016)? Moreover, it seems reasonable to argue also with the late Foucault against a necessary outcome in favor of the existing hegemony circumstances. If we do so the “tool of power” can become a tool of the people.
With the questioning quantifying measurement of social realities might Mihaljević et al. overreach the point. Actually, it is not clear to me, if they want us to stop such kind of analyses at all or if they just aim for the necessary interaction with qualitative interpretations and open science practice. On the one hand, they question the necessity of any gender related data (Mihaljević et al., 3.) At the same time they claim, data should be described with quantitative and qualitative information in order to guarantee reproducibility and transparency. Here the authors refer to the study of Ridge 2015 and underline it with a formalized framework of Gebru et al. 2018. The latter statement seems also reasonable to me. This is also why Mihaljević et al. strongly emphasize the importance of open science and a transparent way of publishing the conditions of a study.
Accordingly, Yves Gingras declared bibliometric analyses as a “tool for historians and sociologist of science” (Gingras 2014, 14) that show the need for critical interpretations that is a basis expertise in these fields. Yet, a critical mind is always needed.
Since Mihaljević et al. “try to foster a debate on best practices for subsequent projects” (Mihaljević et al., 1.) also new approaches seem to be appreciated by them. Also, the need for a stable solution to problematic data in bibliographical analyses is expressed by the authors (Mihaljević et al., 2.1). The data of Wikidata can be a good starting point for that. Of course only with a clear declaration of the research conditions and with a critical interpretation.
Actually, this is, what we try to develop with “Take it Personally”. In any case we expect an intensive inspection of our approach.
Gingras, Yves (2014): Bibliometrics and research evaluation. Uses and abuses, Cambridge (MASS).
Mihaljević, Helena/ Tullney, Marco, Santamaria, Lucia/Steinfeld, Christian (2019): Reflections on Gender Analyses of Bibliographic Corpora, in: Frontiers in Big Data 2:29, https://doi.org/10.3389/fdata.2019.00029
ORCID: Include Gender, Post in ideas archive, https://support.orcid.org/hc/en-us/community/posts/360016228173-include-gender
Seidlmayer, Eva/Galke, Lukas/Melnychuk, Tetyana (2019): Take it Personally – A Python library for dataenrichment in informetrical applications, conference proceedings of Semantics 2019, http://ceur-ws.org/Vol-2451/paper-23.pdf.
Wilkinson, Mark et al. (2016): The FAIR Guiding principles for scientific data, management and stewardship, in: Scientific Data 3, article number 160018, doi: 10.1038/sdata.2016.18.