Finding BERT’s Idiomatic Key

Vasudevan Nedumpozhimana; John Kelleher

doi:10.18653/v1/2021.mwe-1.7

Finding BERT’s Idiomatic Key

Research output: Contribution to conference › Paper › peer-review

Abstract

Sentence embeddings encode information relating to the usage of idioms in a sentence. This paper reports a set of experiments that combine a probing methodology with input masking to analyse where in a sentence this idiomatic information is taken from, and what form it takes. Our results indicate that BERT’s idiomatic key is primarily found within an idiomatic expression, but also draws on information from the surrounding context. Also, BERT can distinguish between the disruption in a sentence caused by words missing and the incongruity caused by idiomatic usage.

Original language	English
Pages	57-62
Number of pages	6
DOIs	https://doi.org/10.18653/v1/2021.mwe-1.7
Publication status	Published - 2021
Event	Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021) - Online Duration: 1 Aug 2021 → 1 Aug 2021

Conference

Conference	Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)
Period	1/08/21 → 1/08/21

Access to Document

10.18653/v1/2021.mwe-1.7

https://aclanthology.org/2021.mwe-1.7.pdfLicence: CC BY

Cite this

@conference{e1c7e4e054c04246a597f7e4fa5b625c,

title = "Finding BERT{\textquoteright}s Idiomatic Key",

abstract = "Sentence embeddings encode information relating to the usage of idioms in a sentence. This paper reports a set of experiments that combine a probing methodology with input masking to analyse where in a sentence this idiomatic information is taken from, and what form it takes. Our results indicate that BERT{\textquoteright}s idiomatic key is primarily found within an idiomatic expression, but also draws on information from the surrounding context. Also, BERT can distinguish between the disruption in a sentence caused by words missing and the incongruity caused by idiomatic usage.",

author = "Vasudevan Nedumpozhimana and John Kelleher",

note = "Publisher Copyright: {\textcopyright} 2021 Association for Computational Linguistics.; Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021) ; Conference date: 01-08-2021 Through 01-08-2021",

year = "2021",

doi = "10.18653/v1/2021.mwe-1.7",

language = "English",

pages = "57--62",

}

TY - CONF

T1 - Finding BERT’s Idiomatic Key

AU - Nedumpozhimana, Vasudevan

AU - Kelleher, John

PY - 2021

Y1 - 2021

N2 - Sentence embeddings encode information relating to the usage of idioms in a sentence. This paper reports a set of experiments that combine a probing methodology with input masking to analyse where in a sentence this idiomatic information is taken from, and what form it takes. Our results indicate that BERT’s idiomatic key is primarily found within an idiomatic expression, but also draws on information from the surrounding context. Also, BERT can distinguish between the disruption in a sentence caused by words missing and the incongruity caused by idiomatic usage.

AB - Sentence embeddings encode information relating to the usage of idioms in a sentence. This paper reports a set of experiments that combine a probing methodology with input masking to analyse where in a sentence this idiomatic information is taken from, and what form it takes. Our results indicate that BERT’s idiomatic key is primarily found within an idiomatic expression, but also draws on information from the surrounding context. Also, BERT can distinguish between the disruption in a sentence caused by words missing and the incongruity caused by idiomatic usage.

UR - https://www.scopus.com/pages/publications/85127799579

U2 - 10.18653/v1/2021.mwe-1.7

DO - 10.18653/v1/2021.mwe-1.7

M3 - Paper

SP - 57

EP - 62

T2 - Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)

Y2 - 1 August 2021 through 1 August 2021

ER -

Finding BERT’s Idiomatic Key

Abstract

Conference

Access to Document

Fingerprint

Cite this