Argument Mining Driven Analysis of Peer-Reviews

The task of peer-reviewing is central to the area of scientific publications. But it is a cumbersome process, due to the growing review workload and newer fields. Argument mining has enabled understanding not only the positions adopted by the people but also the reason why they do so. In the domain of the scientific discourse, specifically in peer-reviewing, this can be potentially used to drive the process.
In our work, we defined the objective to design and develop a reviewing expert system that can assist the editors, meta-reviewers, and reviewers. We demonstrated that the decision process in peer-reviewing is driven by arguments and automatic argument identification can be used to facilitate the process. Within the scope of our project, we defined, structured, and annotated our own scientific-reviews dataset and used it to train our model to extract relevant arguments and then make predictions on the acceptability of the paper. Our machine-learning model achieved near-human performance and showed that arguments are paramount for the publication decision. The process remains interpretable since the extracted arguments can be highlighted in a review without detaching them from their context.

Ressources: Code, Dataset, AAAI-2021 Paper
Team: Siddharth Bhargava, Ruoxia Qi, Yao Zhang, Lukas Dennert, Sophia Selle, Yang Mao
Betreuung: Michael Fromm
Leitung des Innovationslabors: Prof. Dr. Thomas Seidl, Prof. Dr. Bernd Bischl
Infrastruktur: Prof. Dr. Dieter Kranzlmüller
LMU Innovation Lab: