Share this article
With live service games increasingly becoming the norm, the industry has entered a community-driven age. Whether that's fan communities on third-party sites, or communities baked into the fabric of the experience, developers are placing increased emphasis on their importance.
While questions around who is responsible for user safety online often don't have very neat answers, there is a growing consensus that platforms should be held accountable for hosting harmful content. Starting in September 2020, under the EU's Audiovisual Media Services Directive, UK communication service regulator Ofcom will be authorised to fine social media companies and online platforms up to 5% of revenue, or suspend operations, for failure to protect vulnerable users from harmful content.
Earlier this year, the Department for Digital, Media, Culture, and Sport in the UK produced its Online Harms White Paper, which also suggested that social media platforms be held accountable for hosted content, and outlined plans for an independent regulator to manage compliance.
Although the tides have been slowly turning, this is a problem Canadian tech firm Two Hat Security has a long history of tackling. Following a recent partnership with image and video classification software firm Image Analyzer, the two companies will work together to facilitate automatic moderation of live streamed video content "at unprecedented levels."
From left to right: Cris Pikes (Image Analyzer), Chris Piebe (Two Hat), and Carlos Figuieredo (Two Hat)
Two Hat and Image Analyzer are automating the moderation process, allowing individual communities and platforms to set the parameters for acceptable behavior or content, and let the system do the rest. For example, the tech can apparently identify content such as the Christchurch mass shooting, which was streamed over Facebook Live in March this year, and shut it down within seconds.
Speaking with GamesIndustry.biz, Two Hat CEO Chris Priebe says his work tackling harmful online content is deeply personal.
"I was bullied in high school, because I wanted to be different and do my own thing, and I didn't want to fit into the whole crowd, so they bullied me quite extensively, to the point where I had death threats and I had to leave town," he says. "So that gave me a passion for stopping bullying on the internet. I think everyone should be free to share without any harassment or abuse, which is our mission for our company."
The pressure to respond to toxic online content has intensified, as earlier this month another shooting was streamed on Twitch, this time in Germany, which left two people dead. But online communities elsewhere are facing similar problems, as social media becomes infected with extreme content, and fringe elements of gaming communities spread vitriol and hate.
This frequently boils over into industry workers' personal lives, as developers find themselves the target of online abuse, or even cybermobbing. The human cost of online toxicity is immeasurable to both the communities where it proliferates, and individuals actively targeted by it.
"How many billions of dollars are being lost because people quit playing, because they didn't feel welcome?"
Cris Pikes, Image Analzyer
Moderating toxic content isn't casualty-free either; earlier this year an investigation into working conditions at Cognizant-operated Facebook content moderation sites in America revealed staff developing symptoms of post-traumatic stress disorder after extended exposure to extreme graphic content, such as animal cruelty, violence, or child exploitation.
Even removing the human cost of it all, the cynical business case is reason enough to encourage healthy, positive communities. According to figures from Two Hat, people are 300% more likely to never return to games or platforms where they experienced toxic behaviour. Conversely, players are three times more likely to return after positive interactions.
"How many billions of dollars are being lost because people quit playing, because they didn't feel welcome?" says Image Analyzer CEO Cris Pikes. "The longer you stay, potentially the more you're going to pay because you're more invested in the game, so that long tail of building up your love for that game."
It's a position Priebe supports, saying that developers are "putting their time into the wrong thing" when there is a "giant looming sign" that points to problems with the community, rather than the game, limiting developers' potential.
"Stop tweaking your game. I hate your community, go fix your freaking community," says Priebe. "That's what [players] want. And that, I think, will move us from a $100 billion to a $200 billion industry."
"You can't assume that players and users know what is expected of them. So even in that sense, the industry needs to do a better job"
Carlos Figuieredo, Two Hat Security
Last week, Two Hat Security and Image Analyzer hosted a content moderation symposium in London to define and classify online harms, with the aim of tackling online toxicity. While it's a problem that won't be going away anytime soon, there are workable solutions, and plenty of things game developers can be doing in the meantime to help manage their communities.
Carlos Figuieredo is the Fair Play Alliance co-founder, and director of community trust and safety for Two Hat. He says one obvious thing that even large companies have failed to do is establish well-defined community guidelines.
"These serve as the baseline, the fundamental approach for everything else that you do in terms of player behaviour and understanding that player behaviour," he tells us. "So no kidding that we are completely, as an industry, unprepared to deal with threats coming from players on Twitter."
Figuieredo says that developers need to be intentional with how they develop communities. He mentions the recent A/B testing carried out by Twitch, which found that making people manually accept the community guidelines resulted in notable and positive change in general behaviour, just through the mere act of establishing expectations.
"You can't assume that players and users know what is expected of them," says Figuieredo. "So even in that sense, the industry needs to do a better job of really showing what is the expected behaviour, what is the unwanted behaviour, and they do need to enforce something."
With the rapid pace of technological advancement, basic steps like can be profoundly impactful. As Pikes says, it's "bit of an arms race," as companies like Facebook built mammoth platforms that quickly run out of their control.
"We haven't put the same effort into balancing the communities as we have balancing the games"
Chris Priebe, Two Hat Security
"They built this absolute machine," says Pikes, "and they had no idea how big it was going to be... They haven't thought about the securities or the educational pieces that should now be put back in as part of that design. So it's almost a retrospective thought... For us it's about enabling those tools and taking them to a market that has evolved too quickly."
There is a priority gap however, according to Figuieredo, who says there is a "lack of understanding" when it comes to the harm caused by toxic game communities.
"People don't necessarily have good stats or understanding that it affects their business, affects their employees as well," he continues. "How is it affecting their community? What is the user churn? There is a lack of understanding, a lack of white papers and good studies on this."
Priebe adds that companies are failing to adopt viable solutions to these challenges. Part of this he believes falls to an engineering backlog, as devs obsess over in-game balance while inadvertently de-prioritising the existential threat of a toxic community, and how that could significantly shorten a game's lifespan.
Community features like chat and audio are often ill-conceived, says Priebe, and the inclusion of poorly implemented communication tools effectively put a "powerful weapon" in the hands of bad actors.
"They can use it to drive everyone out of playing because they've made it miserable for everyone else," he says. "We haven't put the same effort into balancing the communities as we have balancing the games.... There is [the] technology that is actually available; let's participate in the solution, and let everyone get involved."
With the rise of extremist hate groups using gaming communities as recruitment grounds, there is a pressing need to address the threat, and ensure that vulnerable people in these spaces are protected. As Pikes says, it's about changing the community attitudes.
"Way back when, it was like people felt they had a safe haven where they could go and play a game, and that's what it was all about," he continues. "Whereas now all these far right groups, for example, have found this a potential grooming ground for radicalisation. But if we give [platforms the] tools and enable those companies to understand that the technology is available, those people will move on to another format, they will find another medium, but we've made this one safe and locked that one down."
As the largest entertainment sector in the world, and one which actively encourages online communities, the games industry is under an incredible amount of public scrutiny. The presence of toxic communities is one thing, but a failure to address the problem is arguably much, much worse.
Read the original here:
Two Hat: Stop tweaking your game and start fixing your community - GamesIndustry.biz
- Twin Oaks Intentional Community - Twin Oaks Intentional ... - December 8th, 2016 [December 8th, 2016]
- The Camphill Assocation of North America Communities - December 9th, 2016 [December 9th, 2016]
- Cohousing - Wikipedia - December 11th, 2016 [December 11th, 2016]
- Communes: the pros & cons of intentional community ... - December 21st, 2016 [December 21st, 2016]
- Jewish Intentional Communities Conference - Hazon - December 25th, 2016 [December 25th, 2016]
- Common Fire Beacon-Newburgh | Creating diverse ... - January 2nd, 2017 [January 2nd, 2017]
- Intentional Housing Communities | www.hampshire.edu - February 5th, 2017 [February 5th, 2017]
- A First Gen Lawyer-Turned-Entrepreneur Pioneers New Standards for College Freshmen - Huffington Post - February 7th, 2017 [February 7th, 2017]
- Disparities in minority unemployment targeted by Iowa officials - DesMoinesRegister.com - February 7th, 2017 [February 7th, 2017]
- ACE program benefits low-income communities - Observer Online - February 7th, 2017 [February 7th, 2017]
- Want a happy old age? Get your friends to be your neighbours - Independent Online - February 7th, 2017 [February 7th, 2017]
- Coalition Calls Itself The 'Eyes, Ears & Voice' Of Pittsburgh's Black Community - 90.5 WESA - February 7th, 2017 [February 7th, 2017]
- 'A community remembers' coming to Hesston - Butler County Times Gazette - February 7th, 2017 [February 7th, 2017]
- Krista Tippett February 01, 2017 - America Magazine - February 7th, 2017 [February 7th, 2017]
- Serving the most diverse urban area on the planet - New York Nonprofit Media - February 8th, 2017 [February 8th, 2017]
- To truly serve the public, community stations must apply standards for what's said on-air - Current - February 8th, 2017 [February 8th, 2017]
- Here's what went down at the NYC launch of Ashley Biden's charitable clothing line - Technical.ly - February 9th, 2017 [February 9th, 2017]
- Appalachian's Alternative Service Experience among nation's top 10 higher education institutions for number of programs - Appalachian State University - February 9th, 2017 [February 9th, 2017]
- Pastor: We must build bridges between police and local black communities - Fort Worth Star Telegram (blog) - February 9th, 2017 [February 9th, 2017]
- Chris Wood: Now more than ever localize! - vtdigger.org - February 10th, 2017 [February 10th, 2017]
- A Business Plan for Healthy Communities - Hospitals & Health Networks - February 10th, 2017 [February 10th, 2017]
- The Death of the Ski Bum and Intentional Tourism - The Catalyst - February 11th, 2017 [February 11th, 2017]
- Heroin hits home: Highways provide "easy access" for drug trafficking in Franklin County - Herald-Mail Media - February 11th, 2017 [February 11th, 2017]
- How Anarchists and Intentional Communities Are Reacting to ... - February 11th, 2017 [February 11th, 2017]
- Ohio Continues with Next Phase of InsideOut Initiative to Combat Win-at-All Costs Sports Mentality - Norwalk Reflector - February 13th, 2017 [February 13th, 2017]
- Microsoft Executives to Keynote Summit EMEA 2017 Conference - Yahoo Finance - February 13th, 2017 [February 13th, 2017]
- Marnita's Table set for Wednesday - Daily Globe - February 14th, 2017 [February 14th, 2017]
- David Littlewood, guest columnist: Time to repeal Dodd-Frank Act and free up our community banks - Waco Tribune-Herald - February 15th, 2017 [February 15th, 2017]
- Ithaca organization encourages people to participate in National Random Acts of Kindness Week - The Ithaca Voice - February 15th, 2017 [February 15th, 2017]
- Portland groups form coalition to eradicate hate - KOIN.com - February 15th, 2017 [February 15th, 2017]
- Nash says 'there's more to do' on diversity at State of the County address - Gwinnettdailypost.com - February 16th, 2017 [February 16th, 2017]
- Anson County community meeting to fight poverty planned for Feb. 18 - Ansonrecord - February 16th, 2017 [February 16th, 2017]
- Spreading the Faith: Moving Coins and Moving Communities - Patheos (blog) - February 17th, 2017 [February 17th, 2017]
- If It Walks Like a Duck - ChicagoNow (blog) - February 18th, 2017 [February 18th, 2017]
- Immigrant Round-ups Stir Fears - Consortium News - February 18th, 2017 [February 18th, 2017]
- Pace: What Should I Give Up This Year? - Covington News - February 20th, 2017 [February 20th, 2017]
- J Mase III of #BlackTransMagick seeks to redistribute resources - Daily Illini - February 20th, 2017 [February 20th, 2017]
- South Side getting trauma center, but it'll be far more than just an emergency room - Fox 32 Chicago - February 21st, 2017 [February 21st, 2017]
- St. Louis Park cohousing community welcomes home all ages - Minneapolis Star Tribune - February 22nd, 2017 [February 22nd, 2017]
- The Benedict Option and Rod Dreher's LGBT Challenge - The Atlantic - The Atlantic - February 23rd, 2017 [February 23rd, 2017]
- Mark Sundeen looks for a better way to live - Missoula Independent - February 23rd, 2017 [February 23rd, 2017]
- Cohousing communities gain popularity, including here in Nashville - WKRN.com - February 24th, 2017 [February 24th, 2017]
- The Christian Retreat From Public Life - The Atlantic - February 24th, 2017 [February 24th, 2017]
- New senior living community eyeing Waxahachie - Waxahachie Daily Light - February 24th, 2017 [February 24th, 2017]
- Better health needs a diverse workforce - Greenville Daily Reflector - February 25th, 2017 [February 25th, 2017]
- Businesses: State needs more immigrants - Mankato Free Press - February 25th, 2017 [February 25th, 2017]
- Cohousing communities gain popularity - WDTN - February 25th, 2017 [February 25th, 2017]
- Letters: Dismiss Schimel, others for maps - The Sheboygan Press - February 28th, 2017 [February 28th, 2017]
- Drums, Voices, and Circles - Memphis Democrat - March 1st, 2017 [March 1st, 2017]
- Food: Four Short Talks brings community to the table - Dailyuw - March 2nd, 2017 [March 2nd, 2017]
- Family School rebuts report on lack of diversity - Coastal View News - March 2nd, 2017 [March 2nd, 2017]
- The Wall Street Journal explores trends in Christian community life sort of - GetReligion (blog) - March 2nd, 2017 [March 2nd, 2017]
- Renting land to highest bidder stumbling block for young people looking to start in agriculture - INFORUM - March 3rd, 2017 [March 3rd, 2017]
- Transportation/Traveling While Living Off Grid - Mother Earth News - March 4th, 2017 [March 4th, 2017]
- New School Board President Believes Schools Belong to Communities - The Exponent Telegram (press release) (registration) - March 6th, 2017 [March 6th, 2017]
- Worcester's retiree health costs 'unsustainable' - telegram.com - Worcester Telegram - March 7th, 2017 [March 7th, 2017]
- 12 on Tuesday: Leslie Orrantia - WISC - Channel 3000 - Channel3000.com - WISC-TV3 - March 8th, 2017 [March 8th, 2017]
- By walking the beat, Kalamazoo officers nurture genuine ... - Michigan Radio - March 8th, 2017 [March 8th, 2017]
- Sometimes the Grass Really is Greener - Memphis Democrat - March 8th, 2017 [March 8th, 2017]
- Is Clallam opening the door to tiny houses? | Sequim Gazette - Sequim Gazette - March 8th, 2017 [March 8th, 2017]
- New St. Paul police program aims to mentor recruits - Minneapolis Star Tribune - March 9th, 2017 [March 9th, 2017]
- A New Kind of Homeless Village is Coming to Kenton. It's a Big Deal. - The Portland Mercury (blog) - March 10th, 2017 [March 10th, 2017]
- Why We Need the Benedict Option and How It Doesn't Have to ... - Patheos (blog) - March 10th, 2017 [March 10th, 2017]
- National Expert Shares Thoughts on Environmental Justice - WUWM - March 11th, 2017 [March 11th, 2017]
- The Promise of Paradise features area - 100 Mile House Free Press - March 12th, 2017 [March 12th, 2017]
- Speak out about your experiences - Hibbing Daily Tribune - March 12th, 2017 [March 12th, 2017]
- Trust comes in several varieties - Muncie Star Press - April 8th, 2017 [April 8th, 2017]
- Intentional neighborhoods take root across country - LancasterOnline - April 8th, 2017 [April 8th, 2017]
- my family did the benedict option before it was cool and here's why it doesn't work - Patheos (blog) - April 8th, 2017 [April 8th, 2017]
- Saint Benedict's Mandate - Patheos (blog) - April 8th, 2017 [April 8th, 2017]
- Cohousing Part I: Creating community and reducing social isolation - Michigan State University Extension - April 8th, 2017 [April 8th, 2017]
- Searching for a greater interfaith understanding - Seattle Globalist - June 6th, 2017 [June 6th, 2017]
- The fight for affordable housing in Jefferson Park continues - Chicago Tribune - June 6th, 2017 [June 6th, 2017]
- A 'Justin Option'? Justin Martyr and the Ben-Op - National Catholic Register (blog) - June 6th, 2017 [June 6th, 2017]
- The Groves of Academe: On Keep the Damned Women Out - lareviewofbooks - June 6th, 2017 [June 6th, 2017]
- Curating Community through Intentional Placemaking - Urban Land - June 6th, 2017 [June 6th, 2017]
- Local ties: New tailgate market locations highlight business and community connections - Mountain Xpress - June 7th, 2017 [June 7th, 2017]
- How Power Street Theatre Company is taking on representation in the arts - Generocity - June 7th, 2017 [June 7th, 2017]
- Open house will celebrate Folk Art Guild's 50 years - News - The ... - Penn Yan Chronicle-Express (blog) - June 7th, 2017 [June 7th, 2017]
- Archbishop: In 'post-Christian world' fidelity, charity, truth stand out - CatholicPhilly.com - June 7th, 2017 [June 7th, 2017]