GazPNE2: A general place name extractor for microblogs fusing gazetteers and pretrained transformer models

Abstract

The concept of “human as sensors” defines a new sensing model, in which humans act as sensors by contributing their observations, perceptions, and sensations. This is crucial for the development of Social Internet of Things, which is an integral part of cyber-physical–social systems. Online social media platforms, as the most active places where users act as social sensors, are responsive to real-world events and are useful for gathering situational information in real time. Unfortunately, posts rarely contain structured geographic information, thus hindering their usage for contributing to various challenges, such as emergency response. We address this limitation by introducing a general approach for extracting place names from tweets, named GazPNE2. It combines global gazetteers (i.e., OpenStreetMap and GeoNames), deep learning, and pretrained transformer models (i.e., BERT and BERTweet), which requires no manually annotated data. It can extract place names at both coarse (e.g., city) and fine-grained (e.g., street and POI) levels and place names with abbreviations. To fully evaluate GazPNE2 and compare it with 11 competing approaches, we use 19 public tweet data sets, containing 38 802 tweets and 22 197 places across the world. The results show GazPNE2 achieves a much higher F1 (0.8) than the other approaches. Furthermore, we apply GazPNE2 to three large unannotated tweet data sets related to over 20 crisis events (e.g., coronavirus disease 2019), containing 560 040 tweets. An F1 of 0.84 is achieved on 3000 tweets, which are randomly selected from the three data sets and then manually annotated. Code and data are available on GitHub page: https://github.com/uhuohuy/GazPNE2 .

Zhiyong Zhou

GISer, Postdoc

My research interests include wayfinding and navigation, spatial data science, and GeoAI.