Building a database of political speech does culture matter in charisma annotations?

For both individual politicians and political parties the in- ternet has become a vital tool for self-promotion and the distribution of ideas. The rise of streaming has enabled po- litical debates and speeches to reach global audiences. In this paper, we explore the nature of charisma in political speech, with a view to automatic detection. To this end, we have collected a new database of political speech from YouTube and other on-line resources. Annotation is per- formed both by native listeners, and Amazon Mechanical Turk (AMT) workers. Detailed analysis shows that both la- bel sets are equally reliable. The results support the use of crowd-sourced labels for speaker traits such as charisma in political speech, even where cultural subtleties are present. The impact of these difierent annotations on charisma pre- diction from political speech is also investigated.