Estimating distributed representation performance in disaster-related social media classification

Pallavi Jain; Robert Ross; Bianca Schoen-Phelan

doi:10.1145/3341161.3343680

Estimating distributed representation performance in disaster-related social media classification

Pallavi Jain, Robert Ross, Bianca Schoen-Phelan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper examines the effectiveness of a range of pre-trained language representations in order to determine the informativeness and information type of social media in the event of natural or man-made disasters. Within the context of disaster tweet analysis, we aim to accurately analyse tweets while minimising both false positive and false negatives in the automated information analysis. The investigation is performed across a number of well known disaster-related twitter datasets. Models that are built from pre-trained word embeddings from Word2Vec, GloVe, ELMo and BERT are used for performance evaluation. Given the relative ubiquity of BERT as a standout language representation in recent times it was expected that BERT dominates results. However, results are more diverse, with classical Word2Vec and GloVe both displaying strong results. As part of the analysis, we discuss some challenges related to automated twitter analysis including the fine-tuning of language models to disaster-related scenarios.

Original language	English
Title of host publication	Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019
Editors	Francesca Spezzano, Wei Chen, Xiaokui Xiao
Publisher	Association for Computing Machinery, Inc
Pages	723-727
Number of pages	5
ISBN (Electronic)	9781450368681
DOIs	https://doi.org/10.1145/3341161.3343680
Publication status	Published - 27 Aug 2019
Event	11th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019 - Vancouver, Canada Duration: 27 Aug 2019 → 30 Aug 2019

Publication series

Name	Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019

Conference

Conference	11th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019
Country/Territory	Canada
City	Vancouver
Period	27/08/19 → 30/08/19

Keywords

BERT
ELMo
Text classification
Twitter
Word embedding

Access to Document

10.1145/3341161.3343680

https://arrow.tudublin.ie/scschcomcon/263Licence: CC BY-NC-SA

Cite this

Jain, P., Ross, R., & Schoen-Phelan, B. (2019). Estimating distributed representation performance in disaster-related social media classification. In F. Spezzano, W. Chen, & X. Xiao (Eds.), Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019 (pp. 723-727). (Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019). Association for Computing Machinery, Inc. https://doi.org/10.1145/3341161.3343680

Jain, Pallavi ; Ross, Robert ; Schoen-Phelan, Bianca. / Estimating distributed representation performance in disaster-related social media classification. Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019. editor / Francesca Spezzano ; Wei Chen ; Xiaokui Xiao. Association for Computing Machinery, Inc, 2019. pp. 723-727 (Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019).

@inproceedings{e9d52b2cc26f4e4cb06bb1ff0acf1f5d,

title = "Estimating distributed representation performance in disaster-related social media classification",

abstract = "This paper examines the effectiveness of a range of pre-trained language representations in order to determine the informativeness and information type of social media in the event of natural or man-made disasters. Within the context of disaster tweet analysis, we aim to accurately analyse tweets while minimising both false positive and false negatives in the automated information analysis. The investigation is performed across a number of well known disaster-related twitter datasets. Models that are built from pre-trained word embeddings from Word2Vec, GloVe, ELMo and BERT are used for performance evaluation. Given the relative ubiquity of BERT as a standout language representation in recent times it was expected that BERT dominates results. However, results are more diverse, with classical Word2Vec and GloVe both displaying strong results. As part of the analysis, we discuss some challenges related to automated twitter analysis including the fine-tuning of language models to disaster-related scenarios.",

keywords = "BERT, ELMo, Text classification, Twitter, Word embedding",

author = "Pallavi Jain and Robert Ross and Bianca Schoen-Phelan",

note = "Publisher Copyright: {\textcopyright} 2019 Copyright held by the owner/author(s).; 11th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019 ; Conference date: 27-08-2019 Through 30-08-2019",

year = "2019",

month = aug,

day = "27",

doi = "10.1145/3341161.3343680",

language = "English",

series = "Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019",

publisher = "Association for Computing Machinery, Inc",

pages = "723--727",

editor = "Francesca Spezzano and Wei Chen and Xiaokui Xiao",

booktitle = "Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019",

}

Jain, P, Ross, R & Schoen-Phelan, B 2019, Estimating distributed representation performance in disaster-related social media classification. in F Spezzano, W Chen & X Xiao (eds), Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019. Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019, Association for Computing Machinery, Inc, pp. 723-727, 11th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019, Vancouver, Canada, 27/08/19. https://doi.org/10.1145/3341161.3343680

Estimating distributed representation performance in disaster-related social media classification. / Jain, Pallavi; Ross, Robert ; Schoen-Phelan, Bianca.
Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019. ed. / Francesca Spezzano; Wei Chen; Xiaokui Xiao. Association for Computing Machinery, Inc, 2019. p. 723-727 (Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Estimating distributed representation performance in disaster-related social media classification

AU - Jain, Pallavi

AU - Ross, Robert

AU - Schoen-Phelan, Bianca

PY - 2019/8/27

Y1 - 2019/8/27

N2 - This paper examines the effectiveness of a range of pre-trained language representations in order to determine the informativeness and information type of social media in the event of natural or man-made disasters. Within the context of disaster tweet analysis, we aim to accurately analyse tweets while minimising both false positive and false negatives in the automated information analysis. The investigation is performed across a number of well known disaster-related twitter datasets. Models that are built from pre-trained word embeddings from Word2Vec, GloVe, ELMo and BERT are used for performance evaluation. Given the relative ubiquity of BERT as a standout language representation in recent times it was expected that BERT dominates results. However, results are more diverse, with classical Word2Vec and GloVe both displaying strong results. As part of the analysis, we discuss some challenges related to automated twitter analysis including the fine-tuning of language models to disaster-related scenarios.

AB - This paper examines the effectiveness of a range of pre-trained language representations in order to determine the informativeness and information type of social media in the event of natural or man-made disasters. Within the context of disaster tweet analysis, we aim to accurately analyse tweets while minimising both false positive and false negatives in the automated information analysis. The investigation is performed across a number of well known disaster-related twitter datasets. Models that are built from pre-trained word embeddings from Word2Vec, GloVe, ELMo and BERT are used for performance evaluation. Given the relative ubiquity of BERT as a standout language representation in recent times it was expected that BERT dominates results. However, results are more diverse, with classical Word2Vec and GloVe both displaying strong results. As part of the analysis, we discuss some challenges related to automated twitter analysis including the fine-tuning of language models to disaster-related scenarios.

KW - BERT

KW - ELMo

KW - Text classification

KW - Twitter

KW - Word embedding

UR - http://www.scopus.com/inward/record.url?scp=85078845098&partnerID=8YFLogxK

U2 - 10.1145/3341161.3343680

DO - 10.1145/3341161.3343680

M3 - Conference contribution

AN - SCOPUS:85078845098

T3 - Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019

SP - 723

EP - 727

BT - Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019

A2 - Spezzano, Francesca

A2 - Chen, Wei

A2 - Xiao, Xiaokui

PB - Association for Computing Machinery, Inc

T2 - 11th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019

Y2 - 27 August 2019 through 30 August 2019

ER -

Jain P, Ross R , Schoen-Phelan B. Estimating distributed representation performance in disaster-related social media classification. In Spezzano F, Chen W, Xiao X, editors, Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019. Association for Computing Machinery, Inc. 2019. p. 723-727. (Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019). doi: 10.1145/3341161.3343680

Estimating distributed representation performance in disaster-related social media classification

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this