Improving natural language processing with human data: Eye tracking and other data sources reflecting cognitive text processing

When humans perform everyday tasks like reading, speaking, and writing, they cognitively also complete many of the tasks that natural language processing strives for computers to replicate. The traces of human cognitive processing can be collected in various data sources such as eye tracking during reading, keystroke logs from typing and acoustic cues, where milliseconds matter.

This thesis shows that there is an unused potential for utilizing eye tracking and other data sources reflecting human cognitive processing of text for natural language processing.

This thesis presents several studies where traces of human text processing can be used to improve a broad range of established natural language processing tasks. The tasks span part-of-speech induction, syntactic parsing, sentiment classification, grammatical error detection and detection of abusive language. The thesis furthermore demonstrates some transfer across related languages by using English eye-tracking recordings to improve French part-of-speech induction.

Technology for recording keystroke logs and prosody features is already common. And the recent advancements of low-cost eye tracking technology promise eye-tracking data to be available in larger quantities, also for low-resource languages. Real-world eye-tracking data poses new challenges compared to laboratory data. One study in this thesis presents first evidence that despite the noise and idiosyncrasies, real-world reading data recorded with a consumer-grade eye tracker can be modelled in machine learning models.