Currently, I am testing the library and run into the following performance problems concerning the Wikidata API.
Applying TIP on use case
In the context of my research project Q-Aktiv, together with Tetyana Melnychuk and Lukas Galke, we examine the development of research activities on the chemical lipid “cholesterol”. Analyzing the keywords allocated to papers on cholesterol we aim to follow the development of the research. Early finding indicate an emphasis of publication activities on the topics of cardiovascular diseases and nutrition in the 50ies to the early 70ies, while from the late 70ies on an increasing number of gynecological related key words can be observed, that are allocated to the scientific papers dealing with cholesterol. This finding suggests an increasing interest in gynecological studies from the 70ies till present days.
Deploying “Take it Personally”-Library we aim to understand if the shifting (or expanding) interest is connected to a change (or expansion) of the group of the researchers themselves. Therefore, LIVIVO database including the worldwide leading database for medicine, Medline, supplemented with other Life Science databases, was filtered for scientific articles on cholesterol (indicated by MeSH vocabulary). More than 14 Mio papers from the food and pharma field could be detected in order to monitor the development of cholesterol-papers through time.
Applying the Pubmed-Id used by Medline and DOI identifier we requested the Wikidata-API using SPARQL if the authors of the cholesterol related scientific papers are known in Wikidata. If they are listed, we request the API for the identifier of the specific author(s). In Wikidata this identifier is called Q-Nr/Q-Id.
When confronting the API with 14 Mio Pubmed-Ids and DOI it was noticeable that Wikidata was only able to identify a poor minority of some thousand articles. Splitting the dump in smaller chunks helped already to increase the number fundamentally to more than 400.000 – which is still few for an incoming request of 14 Mio documents. However, the API of Wikidata still delivers unstable results. When requesting the Q-Nr.s of the same initial document the results sometimes deviate in thousands from another.
The performance problems seems to be known in the Wikidata Foundation (mentioned for example in the discussion connected to Taraborelli/Mietchen 2018). We first tried to find a workaround in chunking the input data in smaller units. We also decided to request not more than three queries simultaneously. The purpose was to reach a greater number of results from the single data set, which supplement each other.
Current approach: creating an own Wikidata triple store
Apart from this workaround we focus on building our own triple store for Wikidata with “singularity” to reach faster and comprising information from Wikidata. For all who deal with a similar approach building an own Wikidata triple store I recommend the blogpost of Iazharichir (2018). As we are still in the process I will report soon on the experiences.
Mietchen, Daniel; Taraborelli, Dario (2018): Wikidata, Wikibase, and a federated ecosystem of structured knowledge for open science. figshare. Presentation. https://doi.org/10.6084/m9.figshare.7195358.v3
Iazharichir (2018): Importing Wikidata Dumps — The Easy Part, Blog post 5. 14. 2018, Topicseed, https://topicseed.com/blog/importing-wikidata-dumps
- Forschergeist-Podcast, nr 72 discusses responsibility of computer sciences: https://forschergeist.de/podcast/fg072-verantwortung-in-der-informatik/
- Bruce Schneider, Schneier on Security, https://www.schneier.com/
In German we have a nice saying: “Auf Messers Schneide” reads as “on the (cutting) edge of the knife” or maybe rather on its blade. The phrase indicates a delicate situation. A situation that can turn out in one or the other result, but will never be in between if you cannot stabilize it on the edge. On the very tiny-tiny sharp line that cuts like a ridge in the mountains one slope from the other.
In Philosophy the knife is also the classical example for a dual use problematic: You can stab someone with a knife or you can cut a pineapple with it; the different goals are contradictory as the mountain sides that fall from the ridge. In the fields of natural science and technology the term “dual use” has become mostly known for the development of knowledge that can be applied for military purposes though theses purposes would not be intended by the initial research. In both cases, the knife or the weapons (bringing war or peace), the value of the item does not depend on itself, but on the agent that drives it. Therefore the description of an item and the ethics how to use it need to be differentiated.
Another example for dual or rather triple use is medication or poison or drug. Here we even have different words depending on the use of the same substance: it depends on the quantity if the substance heals or harms. Compared to the knife, one more capacity comes here into the picture of dual use techniques: It’s not only the agent and her motivation, but also the amount or constitution of the item.
We have a similar situation with the data revolution we are experiencing in these times. Data is also an example for a dual use item turning to one or the other side of the edge. And this is why I chose the title “on the edge of the data knife”. The collection of data and the analysis of data can be used in favor of the people as well as against them.
We have seen how protesters organized support structures and demonstrations during the so called “Arabic Spring” 2011 with social networks (Khondker 2011). And we observe similar movements nowadays during the protests in Hongkong in Summer 2019 (Schmidt 2019). On the other hand governments request messenger services to hand out information on communication and connections of their citizens. Luckily, in the case of Iran protests in 2015 the messenger Telegram rejected the attempt by the regime to spy activists (BBC 2015). We also witness how companies like Facebook and Cambridge Analytica use those techniques for personal optimized advertisements and so called “dark ads” (personalized campaigning) that also influenced elections not only in the USA election 2017 but also in India, Kenia and others (Kreysler et al. 2019).
So, how to react to those developments? We could throw away all kinds of consumer cards, user accounts, social network representations, messenger and tracking apps, email-accounts and try to live an analogue life. Frankly, I’m convinced future punks will go further in this direction with all my sympathies. Yet, I don’t believe this will work out if you want to remain a part of mainstream society. We cannot escape the data driven world we just entered. However, within the brave new data world we do not only act as data producers. We can also analyze the data ourselves we cannot run away from.
Because quantitative data analyses render social realities in statistics, rankings, indicators or networks they have to be utilized with caution. Numeric models of reality legitimize political decisions and contribute to the reproduction of social conditions (Angermuller/van Leeuwen 2019). However, if they do so, these models could also justify and substantiate a development towards an emancipatory orientation of society.
Maybe this blog takes too early position. As it also accompanies the development of a data analysis tool it already answered the question if we should stay and try to deal with the data techniques without contributing to the “automation of inequalities” (Eubanks 2018) . “Playing the game” of quantitative techniques in order to understand how they work seems for now to be preferred over backing up off a game we cannot exit. Furthermore, understanding how techniques of data analyzing work is the very condition to criticize and to deal with them deliberately.
Angermuller, Johannes, and Thed van Leeuwen. 2019. “On the Social Uses of Scientometrics: The Quantification of Academic Evaluation and the Rise of Numerocracy in Higher Education.” In Quantifying Approaches to Discourse for Social Scientists, ed. Ronny Scholz, Cham, 89–119.
BBC. 2015. “لگرام ‘پس از عدم همکاری با دولت ایران موقتا مسدود شد’ (Telegram ’temporarily blocked after non-cooperation with Iranian government’).” BBC Persian. https://www.bbc.com/persian/iran/2015/10/151019_u04_telegram_iran (October 8, 2019).
Eubanks, Virginia (2018): “Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor.” New York.
Khondker, Habibul Haque. 2011. “Role of the New Media in the Arab Spring.” Globalizations 8(5): 675–79.
Kreysler, Peter, Kapohl, Matthias, and Schiller, Wolfgang. 2019. “Digitale Brandbeschleuniger Der unregulierte Wahlkampf im Netz.” Deutschlandfunk. https://www.deutschlandfunkkultur.de/der-unregulierte-wahlkampf-im-netz-digitale.3720.de.html?dram:article_id=456530.
Schmidt, Fabian. 2019. “Hongkong: Wie mit Apples AirDrop die Demonstrierenden in Hongkong mobilisiert werden – Gerechtigkeit – Bento.” https://www.bento.de/politik/hongkong-wie-mit-apples-airdrop-die-demonstrierenden-in-hongkong-mobilisiert-werden-a-3075b1aa-06fc-4980-b6c0-e5cdd7f19da7 (October 8, 2019).
Blog post by me on my workplace’s website covering the kickoff event of Free Knowledge Fellowship in September 2019 at Wikimedia in Berlin.
discussion of Mihaljević et al. 2019
The article “Reflections on Gender Analyses of Bibliographic Corpora” by Mihaljević et al. (2019) discusses the problems of a bibliometric analysis of research publications with respect to gender mainly in the field of Science, Technology Engineering and Mathematics. The authors notice the lack of data that currently prevents a decent analysis and request a development of a sustainable solution. In the end the only feasible approach appears to be an open science approach as it can also cover the additionally discussed methodological problems of dealing with the findings of a quantitative analyses of data. Besides this, as their “ultimate goal” the authors depict “to foster a discussion (…) when making statements about individuals and existing societal issues based on publication metadata.” (Mihaljević et al., 1.) Okay, here you go 🙂
In the first place, the authors motivate a high interest in the connection between gender and the under-representation of women in academic positions and publications. They even call it “a natural question” that asks for the contributions of different publication practices of social groups to the observed gender gap (Mihaljević et al., 1.) On the other hand, they state that it is basically not possible to perform a decent analysis due to a lack of personal information in the data sets (Mihaljević et al., 2.1). They also raise methodological concerns in generating these data and point on the danger of misinterpretation of the derived findings leading to an enhancement of inequality when they are taken without qualitative contextualization (Mihaljević et al., 2.3). An earlier finding of the authors is taken as an example, that show that women publish in less prestigious journals (Mihaljević-Brand et al. 2016). This observation could be taken as an argument to rectify why women are less represented in academic positions. Those rush conclusions can be avoided by a careful evaluation of the study and its conditions. To put the challenge in a nutshell: The metadata of publications are agnostic to social aspects as gender. The science system should, but is not.
It is true: the default bibliometric data set, such as MARC21 or as RDA, do not register the gender of authors of a publication. They give a name string that often enough is only one derivate of the name of an individual, due to marriage, multiple first names or transliteration, especially if different alphabets of the original name had been involved (Mihaljević et al., 2.2).
Actually, the gender-blindness (among other blindnesses) of publication metadata is not only a disadvantage: For domain specific investigations, the absence of personal details can be considered as nothing less than a condition for research. Individual information could even corrupt an unbiased view on disciplinary topics. For meta-analyses as we do in bibliometrics, instead, these details would highly improve the understanding of the mechanisms of science as a social field (Seidlmayer et al. 2019). Bibliometric analyses are an important technique for the investigation in the development of science (Gingras 2014, 18).
This distinction between domain specific research questions and the bibliometric meta-questions, might not be taken sufficiently into account by Mihaljević et al. when they generally refuse statements on sensitive topics and “keep questioning the necessity of any given gender-related data analysis” (Mihaljević et. al., 3.). In the context of the given huge social gaps already mentioned, the reaction to the methodological problems should rather claim for a better data basis and a clearly defined usage of the outcomes than to stop the investigation. Data analysis on social circumstances can provide us stable arguments in favor of the marginalized groups of people. In order to ensure a positive outcome the data basis needs to be improved along with the critical handling of the results.
First, we need to adhere something else: If we want to reach a more detailed picture of social privileges and disadvantages we need to broaden the scope of social aspects from gender to many more that lead to disadvantages in academic careers and publication behavior. Besides gender, other social characteristics, such as the social background of the family, the country of origin or the financial power of the institution affecting equipment and paying of publication charges highly influence the research and publication behavior.
Even further go the history of sciences and sociology of sciences, from Ludwik Fleck to Bruno Latour, who discuss the laboratory and the processes within as social shaped – starting with the relations between individuals to the gestures during experiments.
Therefore, a strong interest in a better understanding of the social aspects of science would be appreciated. It is not only gender – although it is considered often as a strong identity aspect, that needs to be taken into consideration to gain a more extensive idea of the complex issue. I guess, also Mihaljević et al. will agree to this broader approach. It comes with similar problems.
However, the main issue for the bibliometric research addressed to publications is a matter of data. The main problem to face is to obtain them. Unsatisfactory are techniques as automated name-allocation of the name-string applying algorithms as Mihaljević conclude convincingly (Mihaljević et al., 2.1) that is offered for example by genderize.io platform. Such approaches of the depiction of gender are justifiably refused as they are a third-party declaration of a very individual characteristic as the gender role. Such techniques can be rejected even more as they perform often in a binary manner. Transgender, inter or other persons will necessarily be misclassified in such heteronormative approaches (Mihaljević et al., 2.1).
The only really convincing method to gain information on issues as gender appears to include the people themselves that are the object of the investigation and ask them on their self-declaration. This comes to the cost of a high effort in time and organization. Furthermore, it needs to be done again and again for every investigation (Mihaljević et al., 2.1).
Wikidata and ORCID as a solution for the missing data?
In the first place, I thought about ORCID and was sure it would solve this. The core concept of ORCID is the self-description that avoids mistakes by third parties assignment of publications due to change of name or other. I was sure that ORCID would be the perfect place for self-declaration and could be used for bibliometric analysis filling up the required missing information. Yet, this is not the case. ORCID does not record such kind of personal information.
Why ORCID deny to register self declarations in the case of gender can be read in the “ideas archive”. Here an interesting conversation on the thread “include gender“ is stored and declared as “not planned”: User Joe Richmond asks to implement gender field since it would help him to contact researchers: “With Chinese, it is often impossible to know and then women are being contacted as if they were men.” Some years later a second user. Lee Raye, a Tutor Swansea University, broadens this argument when she/he/they adds: “In academia we regularly have to talk about people in the third person “Her research suggests that…”. It is not always possible to know someone’s preferred pronoun (e.g. “he”, “she”, “they”). This is not just if the researcher has a gender-neutral name, but also if they are transgender. Having the gender or, especially, a pronoun listed would be very useful.” As the demands aim for a personal contact or an imaginary discussion in an academic text, they are differently motivated than our concern of a bibliographic meta-analysis that looks at the large scale but still convincing matters.
The ORCID-support replied as follows: “[…] ORCID records are specifically designed not to collection personally identifiable information (other than email address). The information on an ORCID record is about the career and research of an individual; ORCID is about linking the researcher to their research-outputs, not providing personal details.”
I understand, that when focusing on the domain specific discussions social aspects should not play a significant role, as we stated above. Although, ORCID might consider not to register academic positions and funding as well if they truly don’t want to publish personal data. In our problem of the missing data for bibliometric studies, ORCID cannot help, at least in the case of gender. But, maybe with information on the career.
At least, the problem of binary gender declaration can be solved easily, as far as I see. Platforms as Wikidata allows the widespread of gender descriptions in as attributes.
In the case of gender for humans, the following Wikidata provides the following items: male (Q6581097), female (Q6581072), intersex (Q1097630), transgender female (Q1052281), transgender male (Q2449503). Those items can still be wrongly related to a person by a third party. The second critic of Mihaljević et al., the self-declaration of people, remains an issue. Most data items on authors in Wikidata are not created by the persons themselves, though, they can be supervised and edited by them.
Therefore, Wikidata might not represent the perfect database for our concern. Though, it is the best we have (as far as I see), if we do not want to cease bibliographic analyses on sensitive social matters.
Quantitative approaches always need a critical inspection
Besides the trouble with the provided data, Mihaljević et al. emphasize a methodological problem concerning the outcome of the data analysis that maps individuals to statistical classes and points on the strong relation to its specific context.
While knowledge relying on an unsatisfactory data basis can only be “approximate knowledge” (Tukey 1962, according to Mihaljević et al., 2.3), Mihaljević et al. go even further and apply this critic to the quantitative analyses itself. Referring to Angermuller and van Leeuwen (2019) they claim, that any quantitative analysis that tries to map is always a factor to the social world it aims to describe (Mihaljević et al., 2.3). They affirm it always reproduces the given circumstances and argue, taken with Foucault, bibliometric research is a mechanism of power.
However, should this critic based on the shoulders of Foucault not pertain to every description and research output? Every interpretation is a single view on a situation. It can always be only a detail and has to be undertaken a critical inspection that includes a reflection on its context. Almost every finding has to be understood as “approximate knowledge” and needs to be discussed related to its conditional statements.
Mihajević et al. claim for reproducibility of research but also emphasize the effects of social characteristics influencing the research. At this point their argument of non-declaration of social characteristics introduced above (Mihaljević et al., 3.) reasoned by its possible transformation into a tool of power (following Foucault) turns to its opposite: For how could we reproduce the research if many influencing aspects – as gender et al. – are not made transparent? The demand for non-declaration rather prevents a reproducibility and does not defend the individual against the statistical arguments.
Does the demand for FAIR principles (findability, accessibility, interoperability, reusebility) not include a self-declaration, as it is common practice in some social science (Wilkinson et al. 2016)? Moreover, it seems reasonable to argue also with the late Foucault against a necessary outcome in favor of the existing hegemony circumstances. If we do so the “tool of power” can become a tool of the people.
With the questioning quantifying measurement of social realities might Mihaljević et al. overreach the point. Actually, it is not clear to me, if they want us to stop such kind of analyses at all or if they just aim for the necessary interaction with qualitative interpretations and open science practice. On the one hand, they question the necessity of any gender related data (Mihaljević et al., 3.) At the same time they claim, data should be described with quantitative and qualitative information in order to guarantee reproducibility and transparency. Here the authors refer to the study of Ridge 2015 and underline it with a formalized framework of Gebru et al. 2018. The latter statement seems also reasonable to me. This is also why Mihaljević et al. strongly emphasize the importance of open science and a transparent way of publishing the conditions of a study.
Accordingly, Yves Gingras declared bibliometric analyses as a “tool for historians and sociologist of science” (Gingras 2014, 14) that show the need for critical interpretations that is a basis expertise in these fields. Yet, a critical mind is always needed.
Since Mihaljević et al. “try to foster a debate on best practices for subsequent projects” (Mihaljević et al., 1.) also new approaches seem to be appreciated by them. Also, the need for a stable solution to problematic data in bibliographical analyses is expressed by the authors (Mihaljević et al., 2.1). The data of Wikidata can be a good starting point for that. Of course only with a clear declaration of the research conditions and with a critical interpretation.
Actually, this is, what we try to develop with “Take it Personally”. In any case we expect an intensive inspection of our approach.
Gingras, Yves (2014): Bibliometrics and research evaluation. Uses and abuses, Cambridge (MASS).
Mihaljević, Helena/ Tullney, Marco, Santamaria, Lucia/Steinfeld, Christian (2019): Reflections on Gender Analyses of Bibliographic Corpora, in: Frontiers in Big Data 2:29, https://doi.org/10.3389/fdata.2019.00029
ORCID: Include Gender, Post in ideas archive, https://support.orcid.org/hc/en-us/community/posts/360016228173-include-gender
Seidlmayer, Eva/Galke, Lukas/Melnychuk, Tetyana (2019): Take it Personally – A Python library for dataenrichment in informetrical applications, conference proceedings of Semantics 2019, http://ceur-ws.org/Vol-2451/paper-23.pdf.
Wilkinson, Mark et al. (2016): The FAIR Guiding principles for scientific data, management and stewardship, in: Scientific Data 3, article number 160018, doi: 10.1038/sdata.2016.18.
Providing and usage of data on people in data science can always be used for conflicting purposes. On the one hand it can provide insight on privileges and discrimination aiming for an empowerment of authors and marginalised groups. On the other hand to process data about people also on large scale carries the danger to be used against them. The blog contains reflections on this controversial aspects of data use from the perspective of an ungoing data science project.