When people act like jerks on social media, one permanent response is to ban them from posting again. Take away the digital megaphone, the theory goes, and the hurtful or dishonest messages from those troublemakers wont post a problem there anymore.
What happens after that, though? Where do those who have been deplatformed go, and how does it affect their behavior in future?
Assistant Professor Jeremy Blackburn, Department of Computer Science, Thomas J. Watson College of Engineering and Applied Science Image Credit: Jonathan Cohen.
An international team of researchers including Assistant Professor Jeremy Blackburn and PhD candidate Esraa Aldreabi from the Thomas J. Watson College of Engineering and Applied Sciences Department of Computer Science explores those questions in a new study called Understanding the Effect of Deplatforming on Social Networks.
The research performed by iDRAMA Lab collaborators at Binghamton University, Boston University, University College London and the Max Planck Institute for Informatics in Germany was presented in June at the 2021 ACM Web Science conference.
Researchers developed a method to identify accounts belonging to the same person on different platforms and found that being banned on Reddit or Twitter led those users to join alternate platforms such as Gab or Parler where the content moderation is more lax.
Also among the findings is that, although users who move to those smaller platforms have a potentially reduced audience, they exhibit an increased level of activity and toxicity than they did previously.
You cant just ban these people and say, Hey, it worked. They dont disappear, Blackburn said. They go off into other places. It does have a positive effect on the original platform, but theres also some degree of amplification or worsening of this type of behavior elsewhere.
More about Blackburn
The deplatforming study collected 29 million posts from Gab, which launched in 2016 and currently has around 4 million users. Gab is known for its far-right base of neo-Nazis, white nationalists, anti-Semites and QAnon conspiracy theorists.
Using a combination of machine learning and human labeling, researchers cross-referenced profile names and content with users that had been active on Twitter and Reddit but were suspended. Many who are deplatformed reuse the same profile name or user info on a different platform for continuity and recognizability with their followers.
Just because two people have the same name or username, thats not a guarantee, Blackburn said. There was a pretty big process of going through creating a ground truth data set, where we had a human say, These have to be the same people because of this reason and that reason. That allows us to scale things up by throwing it into a machine learning classifier [program] that will learn the characteristics to watch for.
The process was not unlike how scholars determine the identity of authors for unattributed or pseudonymous works, checking for style, syntax and subject matter, he added.
In the dataset analyzed for this study, about 59% of Twitter users (1,152 out of 1,961) created Gab accounts after their last active time on Twitter, presumably after their account was suspended. For Reddit, about 76% (3,958 out of 5,216) of suspended users created Gab accounts after their last post on Reddit.
Comparing content from the same users on Twitter and Reddit versus Gab, users tend to become more toxic when they are suspended from a platform and are forced to move to another platform. They also become more active, increasing the frequency of posts.
At the same time, the audience for Gab users content is curtailed by the reduced size of the platform compared to the millions of users on Twitter and Reddit. This might be seen as a good thing, but Blackburn cautioned that much of the planning for the Jan. 6 attack on the U.S. Capitol happened on Parler, a platform similar to Gab with a smaller user base that skews to the alt-right and far-right.
Reducing reach probably is a good thing, but reach can be easily misinterpreted. Just because someone has 100,000 followers doesnt mean theyre all followers in the real world, he said.
The hardcore group, maybe the group that were most concerned about, are the ones that probably stick with someone if they move elsewhere online. If by reducing that reach, you increase the intensity that the people who stay around are exposed to, its like a quality versus quantity type of question. Is it worse to have more people seeing this stuff? Or is it worse to have more extreme stuff being produced for fewer people?
A separate study, A Large Open Dataset from the Parler Social Network, also included Blackburn among researchers from New York University, the University of Illinois, University College London, Boston University and the Max Planck Institute.
Presented at the AAAI Conference on Web and Social Media last month, it analyzed 183 million Parler posts made by 4 million users between August 2018 and January 2021, as well as metadata from 13.25 million user profiles. The data confirm that users on Parler which briefly shut down and was taken off of Apple and Google app stores in response to the Capitol riot overwhelmingly supported President Donald Trump and his Make America Great Again agenda.
Regardless of what Parler might have said, publicly or not, it was very clearly white, right-wing, Christian Trump supporters, Blackburn said. Again, unsurprisingly, it got its largest boost right at the 2020 election up to a million users joining. Then around the attack at the Capitol, there was another big bump in users. What we can see is that it was very clearly being used as an organization tool for the insurrection.
So if banning users is not the right answer, what is? Reddit admins, for example, have a shadow-banning capability that allows troublesome users to think theyre still posting on the site, except no one else can see them. During the 2020 election and the COVID-19 pandemic, Twitter added content moderation labels to tweets that deliberately spread disinformation.
Blackburn is unsure about all the moderation tools that social media platforms have available, but he thinks there need to be more socio-technical solutions to socio-technical problems rather than just outright banning.
Society is now fairly firmly saying that we cannot ignore this stuff we cant just use the easy outs anymore, he said. We need to come up with some more creative ideas to not get rid of people, but hopefully push them in a positive direction or at least make sure that everybody is aware of who that person is. Somewhere in between just unfettered access and banning everybody is probably the right solution.
Read more:
- The Grotesque Slander of Comparing Gaza Protesters to Neo-Nazis - The New Republic - April 29th, 2024 [April 29th, 2024]
- Joe Alt Is A Strong Fit For The Los Angeles Chargers - Sports Illustrated - April 29th, 2024 [April 29th, 2024]
- How The Manosphere Of Alt Right Content Targets Boys - BuzzFeed - April 29th, 2024 [April 29th, 2024]
- Say Hello To Joe Alt, Future Hall Of Fame Right Tackle - Barstool Sports - April 29th, 2024 [April 29th, 2024]
- An Alt-Right Icon Just Gave the Single Dumbest Seinfeld Take Ever - Cracked.com - April 4th, 2024 [April 4th, 2024]
- Ireland & Wales Rugby Fans Take On Alt-Right On Twitter - Balls.ie - February 18th, 2024 [February 18th, 2024]
- Doja Cat Reacts to Backlash Over Alt-Right Comedian T-Shirt: 'It Didn't Affect the World' - TooFab - December 19th, 2023 [December 19th, 2023]
- Doja Cat Responds To Backlash Over Alt-Right T-Shirt: 'It's Not An Attack' - HipHopDX - December 19th, 2023 [December 19th, 2023]
- Marvel and DC Writer Mark Waid Rejects Mark Millar's Call To Root Out Comic Book 'Cancel Pigs', Dishonestly Paints ... - Bounding Into Comics - December 19th, 2023 [December 19th, 2023]
- Maudlin Strangers unveil intense alt rock banger Im Not In The Right Mind [Video] - EARMILK - December 19th, 2023 [December 19th, 2023]
- The Dark Economics of Russell Brand - WIRED - September 23rd, 2023 [September 23rd, 2023]
- Pro-Kremlin Propagandist Ties to White Nationalist Movement ... - Southern Poverty Law Center - September 23rd, 2023 [September 23rd, 2023]
- Opinion | How to Argue Against Identity Politics Without Turning Into ... - The New York Times - September 23rd, 2023 [September 23rd, 2023]
- Anna Wintour: 'I just have to make sure things are being done right' - Financial Times - September 23rd, 2023 [September 23rd, 2023]
- Opinion | Donald Trumps Abortion Shell Game - The New York Times - September 23rd, 2023 [September 23rd, 2023]
- How to Treat Right-Wing Violence in the U.S. - The New Yorker - August 30th, 2023 [August 30th, 2023]
- Scalise, No. 2 House Republican, Says He Has Blood Cancer - The New York Times - August 30th, 2023 [August 30th, 2023]
- Emails reveal Secret Service contacts with Oath Keepers - Citizens for Responsibility and Ethics in Washington - August 30th, 2023 [August 30th, 2023]
- How Trump's Election Lies Left the Michigan G.O.P. Broken and ... - The New York Times - August 30th, 2023 [August 30th, 2023]
- Alt Right: A Primer on the New White Supremacy | ADL - March 2nd, 2023 [March 2nd, 2023]
- The Dangerous Subtlety of the Alt-Right Pipeline - March 2nd, 2023 [March 2nd, 2023]
- The Alt-Right Manipulated My Comic. Then A.I. Claimed It. - March 2nd, 2023 [March 2nd, 2023]
- Richard Spencer: Five Things to Know - Anti-Defamation League - February 20th, 2023 [February 20th, 2023]
- From Alt Right to Alt Lite: Naming the Hate - Anti-Defamation League - February 18th, 2023 [February 18th, 2023]
- Alt-right Definition & Meaning | Dictionary.com - February 18th, 2023 [February 18th, 2023]
- What Is QAnon, the Viral Pro-Trump Conspiracy Theory? - January 8th, 2023 [January 8th, 2023]
- Benedict XVIs Most Powerful Influence on the Catholic Church Came Before He Was Pope - The New Yorker - January 6th, 2023 [January 6th, 2023]
- Netflix Removed Kanye's 'Harmful Alt-Right Beliefs' From David ... - November 23rd, 2022 [November 23rd, 2022]
- Where Are All the Women on This Alt-Right, Anti-Choice, Toxic ... - November 23rd, 2022 [November 23rd, 2022]
- Forbes Gets Punked - Labels FTX Former CEO's Girlfriend a "New Darling ... - November 23rd, 2022 [November 23rd, 2022]
- Alt-Right Picks Wrong Side in Dutch Farm Crisis - November 23rd, 2022 [November 23rd, 2022]
- La Mirada Chamber Invited Alt-Right Racist to Moderate School Board ... - October 30th, 2022 [October 30th, 2022]
- Hate & Extremism | Southern Poverty Law Center - October 30th, 2022 [October 30th, 2022]
- America First Political Action Conference - Wikipedia - October 25th, 2022 [October 25th, 2022]
- Pepe the Frog - ADL - October 25th, 2022 [October 25th, 2022]
- Paul Joseph Watson - Wikipedia - October 25th, 2022 [October 25th, 2022]
- Jack Posobiec - Wikipedia - October 25th, 2022 [October 25th, 2022]
- QUIET - Wikipedia - October 25th, 2022 [October 25th, 2022]
- Shintaro Abe - Wikipedia - October 25th, 2022 [October 25th, 2022]
- #FireMcMaster, explained. How alt-right media and a handful of | by ... - October 23rd, 2022 [October 23rd, 2022]
- Right-wing politics - Wikipedia - October 23rd, 2022 [October 23rd, 2022]
- Real Viking History and the Imagined White Supremacist Past | Time - October 23rd, 2022 [October 23rd, 2022]
- More Toxic, Alt-Right, QAnonism | No Minister - October 23rd, 2022 [October 23rd, 2022]
- Steve Bannon's influence on conservative politics: Expert on alt-right explains - University of Colorado Boulder - October 23rd, 2022 [October 23rd, 2022]
- Alt-right - Wikipedia - October 17th, 2022 [October 17th, 2022]
- Alt Right: A Primer on the New White Supremacy - Anti-Defamation League - October 17th, 2022 [October 17th, 2022]
- How the Alt-Right Happened - American University - October 17th, 2022 [October 17th, 2022]
- Alt-right pipeline - Wikipedia - October 17th, 2022 [October 17th, 2022]
- Richard B. Spencer - Wikipedia - October 6th, 2022 [October 6th, 2022]
- incel Meaning & Origin | Slang by Dictionary.com - October 6th, 2022 [October 6th, 2022]
- red pill Meaning & Origin | Slang by Dictionary.com - October 6th, 2022 [October 6th, 2022]
- Just The News - Media Bias/Fact Check - October 6th, 2022 [October 6th, 2022]
- The Download: TikTok moral panics, and DeepMinds record-breaking AI - MIT Technology Review - October 6th, 2022 [October 6th, 2022]
- Roe vs. Trump in the Michigan Midterms - The New Yorker - October 6th, 2022 [October 6th, 2022]
- PhD student examines clash of masculine identities on internet hate site - University of Toronto - September 27th, 2022 [September 27th, 2022]
- Clickbait extremism, mass shootings, and the assault on democracy time for a rethink of social media? - The Conversation - September 27th, 2022 [September 27th, 2022]
- By Reading Brainwaves, an A.I. Aims to Predict What Words People Listened to - Smithsonian Magazine - September 15th, 2022 [September 15th, 2022]
- A Case of Rhabdomyolysis in a Young, Morbidly Obese, Asthmatic Woman With COVID-19 - Cureus - September 11th, 2022 [September 11th, 2022]
- Mothers of the movement: Leadership by alt-right women paves the way for violence - The Conversation - September 11th, 2022 [September 11th, 2022]
- An antidemocratic philosophy called 'neoreaction' is creeping into GOP politics - The Conversation - August 2nd, 2022 [August 2nd, 2022]
- How Stop the Steal Captured the American Right - The New York Times - August 2nd, 2022 [August 2nd, 2022]
- Alt Right: A Primer on the New White Supremacy - June 24th, 2022 [June 24th, 2022]
- Will Tehama Sheriff Candidates Ties to Alt-Right Extremists Alarm ... - June 24th, 2022 [June 24th, 2022]
- Racist Great Replacement Conspiracy Went From Alt-Right to Mainstream - June 24th, 2022 [June 24th, 2022]
- Trump and his mob tried to overthrow American democracy and threaten to try again - Ohio Capital Journal - June 24th, 2022 [June 24th, 2022]
- Pacifism is the wrong response to the war in Ukraine - The Guardian - June 24th, 2022 [June 24th, 2022]
- How Culture Went Back to the Middle Ages - ArtReview - June 24th, 2022 [June 24th, 2022]
- FoxO3 restricts liver regeneration by suppressing the proliferation of hepatocytes | npj Regenerative Medicine - Nature.com - June 24th, 2022 [June 24th, 2022]
- The Alt-Right Has Its Very Own TV Show On Adult Swim - June 11th, 2022 [June 11th, 2022]
- The Nation: "The Libertarian Party Goes Alt-Right" - June 11th, 2022 [June 11th, 2022]
- Milwaukee business owners don't all want the RNC - Wisconsin Examiner - June 11th, 2022 [June 11th, 2022]
- Alt-Right Incels at Deep State Daily Stormer Site Celebrate Buffalo ... - June 3rd, 2022 [June 3rd, 2022]
- Alt Right Journalist Whos Lost Every Lawsuit Over Banned Accounts ... - June 3rd, 2022 [June 3rd, 2022]
- PBS, Lincoln Project Label DeSantis, Press Secretary 'Alt-Right' - June 3rd, 2022 [June 3rd, 2022]
- Johnny Depp-Amber Heard Verdict: The Actual Malice of the Trial - The New York Times - June 3rd, 2022 [June 3rd, 2022]
- How To Keep Meat Juicy With Science : Short Wave - NPR - June 3rd, 2022 [June 3rd, 2022]
- What Do Female Incels Really Want? - The Atlantic - May 17th, 2022 [May 17th, 2022]
- 'Well be here again': How tech companies try and fail to prevent terrorism - Protocol - May 17th, 2022 [May 17th, 2022]
- Republican and more Republican: Idaho shifts ever rightward - The Guardian - May 11th, 2022 [May 11th, 2022]
- Listen up! Here are the finalists of the 2022 Student Podcast Challenge - GBH News - May 11th, 2022 [May 11th, 2022]