Recent announcements from social media heavyweights suggest the private sector is implementing consequences for “bad” behavior across the board, and this trend is likely to spread.

While content moderation has been around as long as blogs, site administrators and websites, big data and AI are unlocking a new kind of content moderation that transcends the previous paradigm. Previously, platforms would process reports of behavior that violated their terms and conditions, which mainly sought to curb pornographic or violent material from being shared. Now, the terms are being updated to cover egregious, racist and hateful speech ― words which garner less consensus in society than pornographic and violent. In addition to more fungible terms of use agreements, Twitch is now considering what you say elsewhere on the web when deciding whether a user should be allowed to use their platform and that is the unique policy which, if adopted elsewhere, will usher in a new paradigm of social moderation.

It is estimated that 90 percent of the world’s data was generated in the last two years, and all that data is being utilized by the public and private sector to serve personalized ads, reduce traffic congestion and track hate speech online, as well as more mild constituent sentiment. Depending on which vantage point one sits, big data is going to save the world or ruin it. Personally, I am ambivalent, but data collaboratives, where private sector entities, government, and civil society come together to share data and problem solve society’s ills is our present and future. APCO Insight research found that D.C. policy influencers and Bay Area tech professionals agree, and by a large margin, that the technology industry is best able to help solve the problems facing the United States. These two trends, the increasing accountability Americans expect from companies and the shared desire for technology companies to partner with government and civil society are enforcing social moderation.

China offers an example of these two forces working together. In 2015, via Alipay someone could order breakfast, make car insurance payments, schedule doctor’s appointments and pay for electricity, gas and the internet. This is the climate which birthed Zhima Credit. Zhima Credit tracks user data, if they allow it, and more than 200 million people have, to produce a score between 350 and 950. The higher the score, the more perks the user receives, such as no down-payments when renting a bicycle, hotel room or umbrella. Formerly, one could skip the security line at the airport with a high enough score. The app looks at what one buys, what degrees a person holds, the scores of friends and many other data points to achieve their result. Ant Financial, the parent company of Alipay, is private and the Zhima Credit score program is voluntary; however, Ant Financial is helping the government to build a social integrity system and Lucy Peng, the executive chair of Ant Financial, was quoted saying Zhima Credit will “ensure that the bad people in society don’t have a place to go, while good people can move freely without obstruction.” Thus, completing the transition from content moderation to social moderation.

America is unlikely to follow the path of China; however, CEOs and other leaders should understand they are operating within a new paradigm of greater social engagement where they are expected to lead initiatives that are moving society in the “right” direction.

Michael Galfetti
Michael Galfetti

Michael Galfetti is a project assistant in APCO Worldwide’s Washington-based public affairs practice, where he conducts research and assists international clients. Read More