Clare Llewellyn and Laura Cram
As part of the Imagine Europe project, Clare Llewellyn and Laura Cram explore the strengths and limitations of using hashtags to determine sentiment in Tweets on the UK’s EU membership.
© iStock.com/AntiMartina. Reproduced under license
In this article, we use our Imagine Europe dataset to analyse the sentiment of hashtags in Tweets on the UK’s EU membership. We look at sentiment in general and, in particular, the difficulty in measuring it. In previous articles, we have discussed in detail the hashtags used by groups campaigning for the UK to either remain in or to leave the European Union (@StrongerIn, @vote_leave, @LeaveEUOfficial).
One of the aspects we want to address in our work is the identification of sentiment in our dataset. We want to ask questions such as: Are people in the UK positive or negative about the EU? and Can we quantify this opinion by topic and/or geography?
Identifying the target of sentiment expressed in a tweet or a piece of text in general is a hard task. There is software available that measures the strength and direction of sentiment in a segment of text, but it is harder to identify what that sentiment is expressed towards. This is what we call the target of the sentiment and it is difficult to automatically extract this target.
For example, we have the tweet below. We can clearly see that this tweet is pro-Remain. If we use a general sentiment analysis tool, the text from this tweet would have a positive sentiment score based on the language used. We can see that there are two targets in this sentence which both have positive sentiment – Sir John Major who ‘is clear’ and Britain which ‘is stronger in Europe’.
http://twitter.com/StrongerIn/status/677080582971592704
After the identification of sentiment polarity, strength and target, we would aim to combine this information to see how these relate to more general topics of discussion and to the relationship between the UK and the EU. In the tweet above, even though a pro-Remain sentiment is expressed, it is difficult to automatically relate the positive sentiment towards the UK and a pro-Remain point of view.
With this tweet, we can use further context and qualifying information, such as the hashtag – but not all Tweets contain this information. Of course, not all EU-related tweets have a stance on the future of the UK-EU relationship. We are currently looking into ways of addressing this issue in more detail and will report our results at a later date.
To present an initial overview of sentiment in the data collected, we are calculating the percentage of hashtags in the data which express a polarity of either pro-Leave or pro-Remain.
We launched the Neuropolitics Research Labs website on 1 December 2015, where you can find more information on our work.
The Neuropolitics Research Lab produces transdisciplinary research, utilising developments in the cognitive neurosciences, to shed new light on political attitudes, identities and decision-behaviours. Our aim is to test the utility of methods more typically associated with neuroscience, informatics and cognitive psychology in helping us to understand more about political attitudes and behaviours.
As part of this site, we have launched the interactive #ImagineEurope Twitter Demonstrator. This provides access to visualisations of the Twitter data we have collected since 7 August 2015.
Social media is being used to monitor ongoing shifts in public imaginings of the European Union at this critical time. Here Twitter is used to track current trends using advanced Twitter analytics, hashtag tracking, sentiment scoring (indicating the rising and falling emotional content of tweets) and trend analysis in response to emerging events.
On the #ImagineEurope Twitter demo there is a sentiment dial – a map showing locations extracted from Tweets and a wordle of commonly used hashtags. This is an overview of the entire dataset. Further pages can be linked to that show sentiment, locations and hashtag wordles by day.
The dataset presented in our interactive Twitter demonstrator is gathered using a set of hashtags related to the upcoming referendum on the UK’s EU membership. We discussed how we collect the data in a previous article. Since then, we have added new hashtags to reflect the ongoing discussion and those used by the Leave and Remain campaigns. These are: #migrant, #refugee, #strongerin, #leadnotleave, #voteremain, #britainout, #leaveeu, #voteleave, #beleave, #loveeuropeleaveeu.
Building a Twitter Dataset to Find out How People View the EU
We have initially adopted the approach of counting the hashtags in our dataset which clearly have a pro-Leave or pro-Remain bias. The Remain hashtags used in the sentiment calculation are: #yes2eu, #yestoeu, #betteroffin, #votein, #ukineu, #bremain, #strongerin, #leadnotleave, #voteremain. The Leave hashtags are: #brexit, #no2eu, #notoeu, #betteroffout, #voteout, #britainout, #leaveeu, #loveeuropeleaveeu, #voteleave, #beleave.
We can see from the dial that most of the data in our sets contains Leave hashtags. We can break this down to show the counts per hashtag.
This breakdown shows that the #brexit hashtag is the most used. This hashtag is not always used to signal support of #brexit. We have discussed in a previous article how in fact this hashtag is used by both pro-Leave and pro-Remain groups. #Brexit appears to be used as a label of the referendum discussion in general rather than an indicator of sentiment direction.
To move beyond the current solution, we are working with The TAG Research Laboratory at the University of Sussex. The Laboratory is involved in research into social media analysis. We are working with them to adapt their software to our data to find a solution to this tricky sentiment problem.
TAG Research Laboratory. In Summer 2010, Jeremy Reffin, and I co-founded the Text Analytics Group (TAG) Research Laboratory. We are currently a team…
Our #ImagineEurope project is part of the Economic and Social Research Council’s The UK in a Changing Europe programme. Look out for our regular updates as the project tracks developments in the debate on the UK’s membership of the EU and follow us on Twitter @myimageoftheEU for more information on this and other projects.
Laura Cram is Senior Fellow, The UK in a Changing Europe, investigating The European Union in the Public Imagination: Maximising the Impact of Transdisciplinary Insights (ESRC/ES/N003985/1).
This article was originally published on the ImagineEurope Storify.
Clare Llewellyn
University of Edinburgh
Clare Llewellyn is PhD Candidate in Informatics and Research Fellow in the Neuropolitics Research Lab at the University of Edinburgh. Her research focuses on user-generated content on the Internet. Her research interests include social media, big data and text and data analytics.
Laura Cram
University of Edinburgh
Prof Laura Cram is Professor of European Politics at the University of Edinburgh; Senior Fellow, The UK in a Changing Europe; and Academic Editor of European Futures. Her research areas include European public policy, European identity and the neuropolitics of public policy and identity.
Shortlink: edin.ac/1ZMMhuf | Republication guidance
Please note that this article represents the view of the author(s) alone and not European Futures, the Edinburgh Europa Institute or the University of Edinburgh.
This article is published under a Creative Commons (Attribution-NonCommercial-NoDerivatives 4.0 International) License.