Understanding & Countering Online Behaviour

How is the internet used in recruitment, radicalisation, and organisation of threats to national security?

How is the internet used in recruitment, radicalisation and organisation of threats to national security? This Programme develops our understanding of the use of the internet in recruitment, radicalisation and organisation of threats to national security, as well as develop ways in which it can be harnessed to mitigate such threats. Led by Professor Adam Joinson at the University of Bath.

The work in this Programme is centered around understanding and shaping online behaviour in a security context. This work is focused on four strands, alongside inter-disciplinary work connecting networks, finance and online behaviour.

  • Radicalisation Online. Following the completion of an indepth review of the state of the art understanding of radicalisation and the internet, we will conduct studies that seek to identify patterns of increasing radicalisation through language and behaviour.
  • Digital Footprints. A systematic review is currently underway that will provide a complete summary and analysis of the current literature linking online behaviour to individual characteristics.
  • Shaping behaviour online. The focus of this strand is the development of markers to evaluate the impact of online interventions.
  • Interdisciplinary insights. Within this programme we are exploiting the inter-disciplinary nature of the project team (e.g. law, psychology, computer science) to explore ways to deepen our understanding of terrorist financial networks, financial fraud and blockchain, and computational analysis of behaviour (e.g. language and rapport).

Original research

The early focus of the programme was on synthesis of existing knowledge. This is now supplemented by original research. The topics for this work include:

  • Evaluating methods of online influence [Bath]
  • Studies of digital footprints and remote assessment of personality [Bath, Lancaster]
  • Language and engagement in ideological online forums [Bath]
  • Blockchain and trust [UWE / Bath]
  • Methods for characterising typical and atypical social media users [Bath]
  • Language and rapport [Lancaster / Bath].

Case study: Digital footprints

The idea of ‘behavioural residue’ has a long history in psychology, with the traces we leave behind in the physical and virtual environment providing clues to personality, socio-demographics and group and interpersonal dynamics such as agreement and rapport. In a security and intelligence context, being able to interpret the ‘residue’ subjects of interest ‘leave behind’ as they interact in virtual environments can provide important clues about their current state of wellbeing, stress levels, personality, likely responses to outside events, and the dynamics within a group. We are conducting a large scale systematic review to collate and evaluate the evidence-base for connecting digital footprints to individual characteristics, and will then conduct a series of studies to test the review insights in the laboratory.

Case study: Radicalisation and the internet

  • Radicalisation is a complex process that generally involves:
  • Perceived relative deprivation: Feeling unfairly disadvantaged compared to others
  • Resentment: Resenting others and seeking to improve situation
  • Blame: Blaming others for hardship and bonding with likeminded others
  • The violent act: Viewing violence as a necessary means to solve problems.

Propaganda disseminated online can attempt to strengthen these feelings. For example, social media and forums can help individuals to find new groups who can reinforce new ideas and even ‘groom’ new recruits. Anonymity can make it easier to deceive, manipulate and present ideologies in a more favourable light.

Principal Investigator

Professor Adam Joinson

Institutions

University of Bath, UK
University of West England (UWE), UK
Lancaster University, UK

People

Project resources

How is Extraversion related to Social Media Use? A Literature Review

With nearly 3.5 billion people now using some form of social media, understanding its relationship with personality has become a crucial focus of psychological research.

As such, research linking personality traits to social media behaviour has proliferated in recent years, resulting in a disparate set of literature that is rarely synthesised. To address this, we performed a systematic search that identified 182 studies relating extraversion to social media behaviour.

Our findings highlight that extraversion and social media are studied across six areas: 1) content creation, 2) content reaction, 3) user profile characteristics, 4) patterns of use, 5) perceptions of social media, and 6) aggression, trolling, and excessive use.

We compare these findings to offline behaviour and identify parallels such as extraverts' desire for social attention and their tendency to display positivity. Extraverts are also likely to use social media, spend more time using one or more social media platforms, and regularly create content.

We discuss how this evidence will support the future development and design of social media platforms, and its application across a variety of disciplines such as marketing and human-computer interaction.

(From the journal abstract)


Thomas Bowden-Green, Joanne Hinds & Adam Joinson, 2020. How is extraversion related to social media use? A literature review. Personality and Individual Differences. https://doi.org/10.1016/j.paid.2020.110040

Exploring Susceptibility to Phishing in the Workplace

Phishing emails provide a means to infiltrate the technical systems of organisations by encouraging employees to click on malicious links or attachments. Despite the use of awareness campaigns and phishing simulations, employees remain vulnerable to phishing emails. The present research uses a mixed methods approach to explore employee susceptibility to targeted phishing emails, known as spear phishing. In study one, nine spear phishing simulation emails sent to 62,000 employees over a six-week period were rated according to the presence of authority and urgency influence techniques. Results demonstrated that the presence of authority cues increased the likelihood that a user would click a suspicious link contained in an email. In study two, six focus groups were conducted in a second organisation to explore whether additional factors within the work environment impact employee susceptibility to spear phishing. We discuss these factors in relation to current theoretical approaches and provide implications for user communities.

Highlights

  • Susceptibility to phishing emails is explored in an ecologically valid setting.
  • Authority and urgency techniques are found to impact employee susceptibility.
  • Context-specific factors are also likely to impact employee susceptibility.
  • A range of targeted initiatives are required to address susceptibility factors.

(From the journal abstract)


Emma Williams, Joanne Hinds, and Adam N. Joinson. 2018. ‘Exploring Susceptibility to Phishing in the Workplace’. International Journal of Human-Computer Studies, 120 (December): 1–13. https://doi.org/10.1016/j.ijhcs.2018.06.004.

Human and Computer Personality Prediction From Digital Footprints

Is it possible to judge someone accurately from his or her online activity? The Internet provides vast opportunities for individuals to present themselves in different ways, from simple self-enhancement to malicious identity fraud. We often rely on our Internet-based judgments of others to make decisions, such as whom to socialize with, date, or employ. Recently, personality-perception researchers have turned to studying social media and digital devices in order to ask whether a person’s digital traces can reveal aspects of his or her identity. Simultaneously, advances in “big data” analytics have demonstrated that computer algorithms can predict individuals’ traits from their digital traces. In this article, we address three questions: What do we currently know about human- and computer-based personality assessments? How accurate are these assessments? Where are these fields heading? We discuss trends in the current findings, provide an overview of methodological approaches, and recommend directions for future research.

(From the journal abstract)


Joanne Hinds and Adam Joinson. 2019. ‘Human and Computer Personality Prediction From Digital Footprints’. Current Directions in Psychological Science, https://doi.org/10.1177/0963721419827849.

What Demographic Attributes Do Our Digital Footprints Reveal?

To what extent does our online activity reveal who we are? Recent research has demonstrated that the digital traces left by individuals as they browse and interact with others online may reveal who they are and what their interests may be. In the present paper we report a systematic review that synthesises current evidence on predicting demographic attributes from online digital traces. Studies were included if they met the following criteria: (i) they reported findings where at least one demographic attribute was predicted/inferred from at least one form of digital footprint, (ii) the method of prediction was automated, and (iii) the traces were either visible (e.g. tweets) or non-visible (e.g. clickstreams). We identified 327 studies published up until October 2018. Across these articles, 14 demographic attributes were successfully inferred from digital traces; the most studied included gender, age, location, and political orientation. For each of the demographic attributes identified, we provide a database containing the platforms and digital traces examined, sample sizes, accuracy measures and the classification methods applied. Finally, we discuss the main research trends/findings, methodological approaches and recommend directions for future research.

(From the journal abstract)


Joanne Hinds and Adam N. Joinson. 2018. ‘What Demographic Attributes Do Our Digital Footprints Reveal? A Systematic Review’. PLOS ONE, 13 (11): e0207112. https://doi.org/10.1371/journal.pone.0207112.

Same Kind of Different: Affordances, Terrorism, and the Internet

The rapid development of the Internet as a cornerstone of private and social life has provoked a growing effort by law enforcement and security agencies to understand what role the Internet plays in terrorism. Paul Gill, Emily Corner, Maura Conway, Amy Thornton, Mia Bloom, and John Horgan's (2017, this issue) effort to identify empirically when and how terrorists engage with the Internet is thus timely and important.

Understanding when terrorists use the Internet is valuable for investigators who must evaluate the immediacy of the risk posed by a suspect or cell. Knowing the typical patterns of use (or lack of use) can facilitate inferences about a cell's preparedness, the nature of its support, and even the goal of its attack. Understanding how terrorists use the Internet is essential for policy makers who must construct legislation to deter citizens from terrorism while retaining their rights to freedom.

This is arguably best accomplished by legislation targeted at a narrow set of Internet uses that are, as far as possible, exclusively associated with illegal actions.

In this policy essay, we focus on two of Gill et al.’s (2017) main contributions. We argue that, subject to robust independent replication, they encourage thought about the functions of the Internet for terrorists, which in turn may have implications that offer useful guidance for policy and practice. Alongside the article's conceptual contributions, Gill et al. also assert to have resolved several pragmatic challenges and we suggest ways in which their solutions, if developed fully, could offer value to the security analyst community. Finally, we take stock of where Gill et al.’s contribution has left us and review the next steps.

(From the journal abstract)


Taylor, Paul J., Donald Holbrook, and Adam Joinson. 2017. ‘Same Kind of Different’. Criminology & Public Policy 16 (1): 127–33. https://doi.org/10.1111/1745-9133.12285.

Out with the Old and … In with the Old? A Critical Review of the Financial War on Terrorism on the Islamic State of Iraq and Levant

This article critically considers the effectiveness of the ‘Financial War on Terrorism’ on the funding streams of the Islamic State of Iraq and Levant (ISIL).

The next section of the article highlights how the international community concentrated on tackling money laundering prior to the terrorist attacks in September 2001 and how this policy dramatically altered. In particular, this section concentrates on the development of and definition of the ‘Financial War on Terrorism’.

The final part of the article seeks to determine if the ‘Financial War on Terrorism’ is able to tackle the funding streams of ISIL.

(From the journal abstract)


Ryder, Nicholas. 2018. ‘Out with the Old and … In with the Old? A Critical Review of the Financial War on Terrorism on the Islamic State of Iraq and Levant’. Studies in Conflict & Terrorism 41 (2): 79–95. http://eprints.uwe.ac.uk/28343/.

Technology and the Formation of Socially Positive Behaviours

‘Behaviour change’ has become a buzz phrase of growing importance to policymakers and researchers. There is an increasing focus on exploring the relationship between social organisation and individual action, and on intervening to influence societal outcomes like population health and climate change. Researchers continue to grapple with methodologies, intervention strategies and ideologies around ‘social change’.

Multidisciplinary in approach, this important book draws together insights from a selection of the principal thinkers in fields including public health, transport, marketing, sustainability and technology. The book explores the political and historical landscape of behaviour change, and trends in academic theory, before examining new innovations in both practice and research. It will be a valuable resource for academics, policy makers, practitioners, researchers and students wanting to locate their thinking within this rapidly evolving field.

(From the publisher's description)


Joinson, Adam, and Lukasz Piwek. 2016. ‘Technology and the Formation of Socially Positive Behaviours’. In Beyond Behaviour Change - Key Issues, Interdisciplinary Approaches and Future Directions, edited by Fiona Spotswood. Bristol: Policy Press. Accessed 20 July 2018. https://policy.bristoluniversitypress.co.uk/beyond-behaviour-change.

Individual Differences in Susceptibility to Online Influence: A Theoretical Review

Scams and other malicious attempts to influence people are continuing to proliferate across the globe, aided by the availability of technology that makes it increasingly easy to create communications that appear to come from legitimate sources. The rise in integrated technologies and the connected nature of social communications means that online scams represent a growing issue across society, with scammers successfully persuading people to click on malicious links, make fraudulent payments, or download malicious attachments.

However, current understanding of what makes people particularly susceptible to scams in online contexts, and therefore how we can effectively reduce potential vulnerabilities, is relatively poor. So why are online scams so effective? And what makes people particularly susceptible to them? This paper presents a theoretical review of literature relating to individual differences and contextual factors that may impact susceptibility to such forms of malicious influence in online contexts.

A holistic approach is then proposed that provides a theoretical foundation for research in this area, focusing on the interaction between the individual, their current context, and the influence message itself, when considering likely response behaviour.

(From the journal abstract)


Williams, Emma J., Amy Beardmore, and Adam N. Joinson. 2017. ‘Individual Differences in Susceptibility to Online Influence: A Theoretical Review’. Computers in Human Behavior 72 (July): 412–21. https://doi.org/10.1016/j.chb.2017.03.002.

Press Accept to Update Now: Individual Differences in Susceptibility to Malevolent Interruptions

Increasingly, connected communication technologies have resulted in people being exposed to fraudulent communications by scammers and hackers attempting to gain access to computer systems for malicious purposes. Common influence techniques, such as mimicking authority figures or instilling a sense of urgency, are used to persuade people to respond to malevolent messages by, for example, accepting urgent updates. An ‘accept’ response to a malevolent influence message can result in severe negative consequences for the user and for others, including the organisations they work for.

This paper undertakes exploratory research to examine individual differences in susceptibility to fraudulent computer messages when they masquerade as interruptions during a demanding memory recall primary task compared to when they are presented in a post-task phase. A mixed-methods approach was adopted to examine when and why people choose to accept or decline three types of interrupting computer update message (genuine, mimicked, and low authority) and the relative impact of such interruptions on performance of a serial recall memory primary task.

Results suggest that fraudulent communications are more likely to be accepted by users when they interrupt a demanding memory-based primary task, that this relationship is impacted by the content of the fraudulent message, and that influence techniques used in fraudulent communications can over-ride authenticity cues when individuals decide to accept an update message. Implications for theories, such as the recently proposed Suspicion, Cognition and Automaticity Model and the Integrated Information Processing Model of Phishing Susceptibility, are discussed.

(From the journal abstract)


Williams, Emma J., Phillip L. Morgan, and Adam N. Joinson. 2017. ‘Press Accept to Update Now: Individual Differences in Susceptibility to Malevolent Interruptions’. Decision Support Systems 96 (April): 119–29. https://doi.org/10.1016/j.dss.2017.02.014.

Radicalization, the Internet and Cybersecurity: Opportunities and Challenges for HCI

The idea that the internet may enable an individual to become radicalized has been of increasing concern over the last two decades. Indeed, the internet provides individuals with an opportunity to access vast amounts of information and to connect to new people and new groups.

Together, these prospects may create a compelling argument that radicalization via the internet is plausible. So, is this really the case? Can viewing ‘radicalizing’ material and interacting with others online actually cause someone to subsequently commit violent and/or extremist acts? In this article, we discuss the potential role of the internet in radicalization and relate to how cybersecurity and certain HCI ‘affordances’ may support it.

We focus on how the design of systems provides opportunities for extremist messages to spread and gain credence, and how an application of HCI and user-centered understanding of online behavior and cybersecurity might be used to counter extremist messages.

By drawing upon existing research that may be used to further understand and address internet radicalization, we discuss some future research directions and associated challenges.

(From the journal abstract)


Hinds, Joanne, and Adam Joinson. 2017. 'Radicalization, the Internet and Cybersecurity: Opportunities and Challenges for HCI'. In Human Aspects of Information Security, Privacy and Trust, 481–93. Lecture Notes in Computer Science. Springer, Cham. https://researchportal.bath.ac.uk/en/publications/radicalization-the-internet-and-cybersecurity-opportunities-and-c

Back to top