Social Media Monitor Toolkit

In the last years we have observed increasingly the risks of democratic discourse manipulation on social media. With advancements in technologies, we observe various online phenomena such as information manipulation and online violence taking place online. The Digital Democracy Monitor Toolkit helps civil society, journalists, researchers and anyone trying to research social media and democracy. Use this toolkit to get started with your own monitoring.

Preparation

Why monitor social media?

Preparation

Getting started in your own context

Preparation

Build your own methodology

Data analysis

How you can access data

Data analysis

Learn tools and techniques

Reporting

Make an impact with social media monitoring

Why monitor Social Media

Make sense of information manipulation, online violence, political advertisements, false amplification methods, influence operations and their influence on political discourse.

Media manipulation doesn’t justhappen on election day…

Although it’s intuitive to focus on election day, manipulation strategies may influence and deceive the public around the clock. Select one of the iceberg sections to know more. 

…and it is more likely to happen around the clock.

Domestic and foreign actors act around the clock to manipulate political discourse. When are they most likely to strike?

This is how media manipulators work

So what aspects can you monitor?

Messenger

Refers to the sender/origin of the message

Exposing who is behind the spread of false information is usually not possible given that users can easily hide behind anonymous accounts. However, there is the possibility to monitor a set of accounts revealing networks and coordination. Coordinated inauthentic behavior (CIB) is a manipulative communication tactic that uses a mix of authentic, fake, and duplicated social media accounts.

Message

Refers to the content of the message

Information Manipulation

Information manipulation encompasses the deliberate dissemination of harmful but accurate information (malinformation), the unintentional spread of false data (misinformation), and the intentional use of false or misleading information to deceive (disinformation).

Online Violence

Refers to concepts such as hate speech and gender-based online violence. Although hate speech is not a universally defined term, it may be understood as messages with the purpose of attacking a person/group based on attributes such as race, religion, ethnic origin, sexual orientation, disability or gender. Gender-based online violence refers to all sorts of illegal or harmful behaviours against women in the online space.

Messaging

Refers to the for of distribution of the message

Advertisement

Paid advertising is a means for actors to boost their message through official methods. Platforms have varying requirements for actors to distribute political advertisements (I.e. additional verification steps).

Attention Manipulators (i.e. Bots)

Bots are automated actors that may be used at scale to spread and amplify false or intimidating messages. They may be used to manipulate the credibility of people or issues and skew online debates.

Influence Operations

Coordinated efforts to manipulate or corrupt public debate for a strategic goal. These efforts can be undertaken directly by state or non-state actors.

WORLD VIEWS
1 Explanation:
Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.
In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.  

 
What strategies may be used?
  • False or misleading information about voting dates or polling locations
  • Harassment campaigns targeted to a specific group in society to disengage them from voting
  • Messaging that questions the integrity or results of an election

 
Who may be the targeting of such messaging?
  • Registered voters in an election
  • A specific ethnic, religious or political group
  • Winning candidates and parties

 
Strategies and Short-term Impacts:
  • Voter suppression
  • Electoral integrity questions
  • Electoral fraud questions
  • Questioning the results by citizens

 
Examples:
  • In the 2022 elections in Brazil, one of the most widely circulated false contents said that the ballot boxes were already being opened and automatically casting ballots for the left-wing candidate, Lula. (Source: Folha de Sao Paulo)  
  • During the 2023 Slovakian elections, we observed disinformation flooding the online debate ahead of crucial parliamentary election. Two days prior to the voting day, a false audio impersonating a candidate to be discussing how to rig the election, partly by buying votes from the country’s marginalized Roma minority. (Source: Wired) 
  • One day before the 2023 Polish Elections, Poland’s government warned citizens to beware of a disinformation campaign after some people got false messages saying that the ruling party was offering free funerals for pensioners. (Source: AP News) 
POLITICAL BELIEFS

Explanation

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

What strategies may be used?

· False or misleading information about voting dates or polling locations

· Harassment campaigns targeted to a specific group in society to disengage them from voting

· Messaging that questions the integrity or results of an election

Examples

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

ELECTORAL PERIOD

Explanation

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

What strategies may be used?

· False or misleading information about voting dates or polling locations

· Harassment campaigns targeted to a specific group in society to disengage them from voting

· Messaging that questions the integrity or results of an election

Examples

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

ELECTION

Explanation

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

What strategies may be used?

· False or misleading information about voting dates or polling locations

· Harassment campaigns targeted to a specific group in society to disengage them from voting

· Messaging that questions the integrity or results of an election

Examples

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

ELECTIONS

Explanation

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

What strategies may be used?

· False or misleading information about voting dates or polling locations

· Harassment campaigns targeted to a specific group in society to disengage them from voting

· Messaging that questions the integrity or results of an election

Examples

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

DISASTERS

Explanation

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

What strategies may be used?

· False or misleading information about voting dates or polling locations

· Harassment campaigns targeted to a specific group in society to disengage them from voting

· Messaging that questions the integrity or results of an election

Examples

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

POLITICAL EVENTS

Explanation

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.

What strategies may be used?

· False or misleading information about voting dates or polling locations

· Harassment campaigns targeted to a specific group in society to disengage them from voting

· Messaging that questions the integrity or results of an election

Examples

Social media manipulation can target Election Day as a short-term attempt to change voter’s behaviour of whether to go to vote or not, without fundamentally targeting a voters’ opinion on a candidate or party.

In the period right after the end of the vote, a fight for public opinion on the credibility of an election may take place on social media. In many elections, the process and the results may be correctly or falsely questioned.


Getting started in your own context

Understanding your political social media ecosystem is essential before beginning your analysis. What are the critical questions that you should ask?

1. Gather your team

  • Teams require a combination of local political knowledge and some data analysis capacities. Consider teammates with:
    • Political, social science or electoral observation background
    • Data scientists or some familiarity with data analysis using Excel
    • A combination of the above
  • In DRI’s experience, teams consist of around 4-7 members
  • Your team’s capacities (size and expertise) will determine the amount of data you can process
  • Teams can start from scratch with the proper training, leadership and political knowledge

2. Create a project timeline

Your project will include several phases:

  1. Preparation: Creating your team, selecting your sample, getting access to data
  2. Data collection: Gathering selected data from your platform(s) of choice
  3. Data analysis: Classifying posts, analyzing trends and visualizing results
  4. Reporting and outreach: Summarizing findings in a report(s) and possible outreach activities

Here are some sample timelines based on real projects to help you get started with project planning:

See below in section Section 6, “What Timeframe?”, for more details on how to select your overall timeframe.

3. Conduct your risk assessment

Conducting a risk assessment prior to starting your work may help your team understand the current situation and narrow your scope. DRI has developed a Risk Assessment framework. See our User Guide to get started:

Risk Assessment Methodology

4. Research any relevant laws

Does your country have any laws relevant to disinformation or hate speech online? Such information might be useful before beginning your monitoring. Especially when it comes to elections, campaign finance laws related to online political advertisements would be particularly important to understand.

Knowledge Hub

5. Narrow your scope

When getting started, there are many options regarding what can be monitored, but it is not possible to monitor it all. With an ocean of opportunities, you must somehow limit these choices into a smaller selection.

To help you decide, you must keep in mind possible restraints while actively choosing your priorities. For example, if your organizational mandate is online campaign finance transparency, you might want to focus on monitoring political advertisements. You might want to monitor the top three social media platforms in your country but will have to look only at the top posts or add more analysts to your team.

6. Remember your starting points

Once you have selected your platform of interest, there are three main starting points when working with social media data: (a) keywords, (b) public fora, and (c) accounts. This means, at the very beginning you will need to create lists of specific topics and keywords to search for, a list of public discussion forums (for example, Facebook groups), or a list of relevant accounts to monitor.

Keywords
Public fora
Relevant
Actors/accounts

7. Answer the guiding questions

Once you have selected your platform of interest, there are three main starting points when working with social media data: (a) keywords, (b) public fora, and (c) accounts. This means, at the very beginning you will need to create lists of specific topics and keywords to search for, a list of public discussion forums (for example, Facebook groups), or a list of relevant accounts to monitor.

Which phenomena?

Which topics?

Which actors?

Which timeframe?

Which platform?

Some phenomena may be particularly important based on your organizational mandate or a risk assessment of your country context:

  • Information Manipulation
  • Online Violence
  • Political/issue advertisements
  • Influence Operations
  • Coordinated Behaviour

BUILD YOUR OWN METHODOLOGY

Transform your knowledge into action!
Based on the platform, look for sample methodologies to study different phenomena.

1. Information Manipulation

2. Online Violence

3. Political Ads

4. Electoral Monitoring

understanding the quality of information on key political topics

Possible for:

Understanding the quality of news, especially on controversial topics during elections is critical to understand if democratic discourse is being manipulated in any way.

  • 1. Sample selection

    · Select all Facebook groups/pages discussing the election and/or relevant topics (e.g. economy).

    • 1. Generate a general list of the most important 10-20 topics (e.g. election) using local-political knowledge.
    • 2. Make a list of relevant keywords (e.g. #election2020) related to each topic.
  • 2. Gather data

    · Use CrowdTangle’s Historical Data function and filter for posts based on your identified keywords identified in Step 1. Make sure to limit to your specific country. .

    · Define a criteria to limit the number of posts to the most relevant ones. Your team’s limited capacity may not be able to handle all available data.

    • 1. For example, in DRI’s 2019 Sri Lankan presidential report we looked at:
      The 250 posts with the highest number of total interactions for pages, and the top 100 posts for groups were chosen from each of the issues monitored. For reference, this resulted in 2,362 posts from pages and 1,000 posts from groups over a two month period, which were analyzed by a team of 7.
  • 3. Classify data (if you have time)

    · Label each post: (1) verified mainstream news (0) non-verified news.

    · Check and label the post for further information (i.e sentiment, specific narrative).

    • 1. Sentiment: Manually labels posts (a) very positive (b) positive (c) neutral (d) negative (e) very negative. Define a clear codebook of what each category means. Depending on your language and coding capacities, you may be able to use R or Python to run an automated sentiment analysis.
    • 2. Narratives: Manually code each post based on topic or simply filter based on keywords defined in Step 1

HOW CAN YOU ACCESS DATA

You need access to social media to conduct your analysis.
Find step-by-step guidance on accessing data from social media platforms.

1. EACH COMPANY HAS DIFFERENT PROCESSES, WHICH RANGE IN DIFFICULTY AND TIME NEEDED TO ACCESS DATA

Because social media research is powered by data, understanding what to access and where is rarely an easy task. Here are some tips:

  • Try to apply for access early in case the process takes longer than planned.
  • If you are having difficulties getting access to data, reach out to a real company employee.
  • Test Application Programming Interfaces (APIs) as soon as possible if further technical capacities are needed on your team.

2. USE DATA ETHICALLY BY FOLLOWING PLATFORM POLICIES AND RELEVANT DATA LAWS

See the EU’s Guide on SMM written by DRI for more information on data ethics.

  • Identify whether any aspects of the data pose a ‘risk to users’.
  • Ensure the data is anonymous and untraceable.
  • Collect only necessary data pertaining to the research question.
  • Disclose when and how data is collected.
  • Disclose how data is analysed with a clear replicable methodology.
  • Store data safely to avoid leaks or abuses.
  • Develop and publish a privacy policy with clear communication channels to answer any questions from the public.

3. DATA OVERVIEW PER PLATFORM

Facebook and Instagram

Meta’s platforms share data through a company-owned social media listening tool, CrowdTangle, which provides access to public pages and groups (Facebook) and public accounts (Instagram). Access is granted to vetted academic researchers or civil society organisations. There is no limit on historical data access, but researchers cannot access unverified public accounts with less than 50,000 followers, and have no access to comments under posts on public pages and public groups. Moreover, there isn´t access to new platform features, such as stories.

See available data
TikTok

TikTok currently offers API access only to developers. Such data can be used by researchers, despite the limitations resulting from the fact it was not developed for research purposes. The developer API includes unlimited access to the historical data of public accounts. TikTok recently announced the development of a researcher API for United States-based academic institutions, including data on public profiles and keyword search results.

See available data
Telegram

Telegram, a messenger platform, provides API access only for developers, but this can be used by researchers and offers wide access to data. Features such as public groups or channels allowing one-way messaging to large audiences resemble social media platforms, making Telegram a hybrid of a social media platform and a messenger.

See available data
YouTube

YouTube provides access for researchers and developers through their API. This allows access to all public channels and provides unlimited access to historical data. YouTube recently introduced a new feature to share content, named “shorts”, and access to data for this is not yet available yet through the API.

See available data
X

X had very well-developed access to data both for developers and vetted researchers and, hence, was also accessible through several third-party social media listening tools. After its acquisition by Elon Musk, certain changes were introduced, such as a paywall for developers’ APIs. The monetisation of access to researcher data was also proposed, but the platform decided to postpone this step. Despite the fact that researcher API is still accessible to researchers without a paywall, non-transparent and confusing communication creates obstacles. Moreover, academic API is granted only to academic researchers, which excludes non-academic research organisations, which, as a result, have to use API for developers.

See available data

4. WHAT IS POSSIBLE TO STUDY

Platform Which Instance? API Who has access? Historical Data? Public Spaces not Available Tool
Facebook Public Pages and Groups Yes Platform Approved Researchers + Developers Unlimited Public Accounts, Comments on Public Pages and Groups, Additional Features CrowdTangle / Ad Library
Instagram Public Accounts Yes Platform Approved Researchers + Developers Unlimited Comments on Public Accounts, Additional Features CrowdTangle / Ad Library
YouTube Public Channels Yes Researchers + Developers Unlimited Additional Features on Public Channels YouTube API v.3
TikTok Public Accounts Yes Vetted Developers* Unlimited Comments on Public Accounts, Additional Features TikTok API
X Public Accounts Yes Paid Access Unlimited Audio X API
Telegram Public Channels and Groups Yes Developers Unlimited All public spaces available Telegram API

*US based

To know more visit our Data Access page.

Data Access


LEARN TOOLS AND TECHNIQUES

Use our database to find tools and packages to analyze your social media data.

WHICH SOCIAL MEDIA MONITORING TOOLS ARE BEST FOR YOU?

The answer to this question will depend on a few aspects. First, what are you trying to accomplish in your monitoring? Second, how long will you have to familiarize yourself with new tools? Third, how much do you want to dive into big data?

We have included information about each tool including how you can use it and resources to get started.

FILTER AND SEARCH OUR DATABASE

If you see any tools missing from our database, please contact us so we can add them!


MAKE AN IMPACT WITH SOCIAL MEDIA MONITORING

How can you share your findings for maximum impact? We have created some reporting templates to help you take your research to action.

WHAT ARE YOU TRYING TO ACHIEVE?

There are many ways in which applied research on social media & elections can have an impact. Whether you want to target policy-makers within your country or simply raise awareness of the risks of disinformation to citizens, some choices need to be taken into consideration. By defining what you are trying to achieve, it is easier to identify your primary audience and the format of your reporting.

This question will impact:

  • How to share your findings (privately or publicly)
  • The format of a successful report
  • Frequency of reporting
  • Possible outreach activities

HOW TO SHARE YOUR FINDINGS?

Depending on your organizational mandate, you may decide to share your findings privately to maintain a low-profile and avoid excessive media attention. In this case, there are a number of steps that you can take.

Make an impact in private:

  • Share findings with authorities to assess electoral violations (if appropriate and safe)
  • Share findings with social media company representatives directly (contact via email)
  • Support investigations on human rights violations (when monitoring hate speech)
  • Share findings with policy-makers to inform legislative debates

On the other hand, you might want to share your findings publically with policy-makers and citizens. If so, you can publish regular reports and consider conducting outreach activities.

Make an impact in public:

  • Publish regular reports to track developments of social media manipulation around elections
  • Collaborate with news organizations to publish your findings and reach more audiences
  • Share findings with international media outlets
  • Conduct outreach activities such as trainings and workshops to present your findings to appropriate audiences
  • Collaborate with think tanks and academia to strengthen research

HOW TO WRITE A SUCCESSFUL REPORT?

Considering your audience, your will want to tailor your report accordingly. Successful reporting will address the intended audience in a concise manner.

Most likely your report will be directed to a non-technical audience interested in the impact of social media in elections. As a result, findings should be explained in language that a broader audience can understand. You can also consider publishing one-pagers with your most important findings to target a policymaking audience with data visualizations. Most importantly, remember to prioritize your findings and tailor your recommendations specifically to your stakeholder audience. Your report should be easy to read but also methodologically transparent to ensure the consistency of your results.

Tips:

  • Use language that a broader, non-technical audience can understand
  • Consider publishing a short and long-form report for more and less technical audiences
  • Prioritize your findings and explain why they matter
  • Include specific recommendations shaped to your audience
  • Avoid a long, general introduction re-capping “the problem” non-specific to your context
  • Your methodology should be transparent and accurate but not overly technical
  • Include a codebook in your appendix so your results are replicable

HOW FREQUENTLY TO REPORT?

Organizations new to SMM should keep in mind that the data analysis process is time-consuming. New teams should think twice before committing to a tight public reporting schedule (e.g. weekly) and assess resources and capacity.

Consider:

  • Assess resources available and set a realistic reporting schedule
  • Make sure you leave a buffer in your schedule in the case you run into hiccups with your data

DRI’s recommended reporting schedule during elections:

  1. A risk-assessment prior to the election
  2. A short report before the election on the campaign to date
  3. A brief report several days after the election while media attention is high
  4. An in-depth report a month following the election (max. 15 pages)

If you have programming knowledge, a good idea is to create a dashboard where the data can be updated frequently (weekly). Based on the data from pre-established accounts that are being monitored, it is easier to draft more frequent insights to be used in weekly updates.

HOW TO REACH RELEVANT STAKEHOLDERS?

Think about planning potential activities to engage relevant stakeholders at the beginning of your project. Getting in touch early with other stakeholders may help you improve the quality of your work. Maybe you will get new ideas for what to monitor and prevent overlap with other projects to maximize the overall impact.

Tips:

  • Get in touch early and have a communication strategy
  • Collaborate with media to publish your findings to reach a wider audience
  • Ask your CSO network to share your findings
  • Invite relevant stakeholders to an event to discuss your findings
  • Create an interactive dashboard in addition to your report
  • Discuss results with similar initiatives to learn new research angles and techniques

CONSIDERATIONS DURING ELECTIONS?

Traditional election observers usually publish preliminary findings shortly after voting followed by a more detailed report months later. They want to avoid becoming part of the campaign discussion, which would make them a part of the process they observe. Additionally, a comprehensive assessment of the full electoral process prevents partial findings from becoming obsolete from one day to the next.

In the case of social media monitoring, reports exposing disinformation campaigns, foreign interference or incitement to violence may actually help prevent issues before an election occurs. As a result, it may make sense to issue interim reports prior to elections and a more in-depth report following an election. When publishing before an election, organizations should be prepared to join the political debate.

Organisations may instead inform authorities (where appropriate) and tech companies of their findings without going public. This may help CSOs steer a middle course between silence and joining the political debate. These decisions need to be ultimately made by each organisation based on its mandate and the national context.

Consider:

  • Consider the pros and cons of reporting before vs. after the election
  • Consider what is the impact you wish to have before
  • Bear in mind any existing electoral legislation that may be relevant prior to conducting your research - in many countries there may be restrictions about electoral related publications prior to the voting day.