Banned social media users have long petitioned for reinstatement via informal appeals amplified by prominent supporters. They may be public campaigns, involving supportive hashtags and mass retweets, or private pleas to platform staff known to the banned user or a verified friend. While platforms provide some formal appeal mechanisms, they rarely provide opportunities to solicit support, clarification, or additional context from other users. At scale, reversing an erroneous ban often seems like a matter of making enough noise to demand a second hearing. Sonya Mann presents this phenomenon as pseudo-feudal, highlighting the inherent inequality of informal collaborative appeals.
When regular users run afoul of the algorithm, or are dinged by a moderator, it’s common for them to reach out to accounts with larger followings. The hope is that their pleas will be sent to the top by a sympathetic intermediary. It’s not unlike begging a duke to bring your grievance to the king. Jack Dorsey doesn’t care if some random Twitter account gets shut down, but he might lift a finger if a sufficiently prestigious bluecheck brought it to his attention. Or at least that’s the theory.
As COVID-19 has sent human moderators home and increased platforms’ reliance on algorithms, the value of informal appeals has only increased. This should be of concern to platforms as well as users. Content moderation is a top-down process. Unable to draw upon democratic legitimacy, moderators have increasingly attempted to legitimize their governance by adhering to procedural values. The Santa Clara Principles, a set of commitments intended to provide users with due process endorsed by Facebook, Twitter, and Youtube, requires “meaningful appeals” and “the ability to present additional information.” If, in practice, only the loudest accounts have this ability, users’ expectations of procedural fairness will be frustrated.
Last month, @MENA_conflict, a mid-sized conflict studies and suburban farming account operated by a former infantry marine was banned from Twitter, apparently after being mass-reported by would-be QAnon-hunters, before being eventually reinstated. The account received a deluge of support from journalists and prominent users, including Jack McCain. Its operator reported that he returned to “like 2500 notifications from y’all harassing twitter to reinstate me.”
Informal appeals may have saved @MENA_Conflict in this case, but they present a broader problem. Platform responsiveness to such mass appeals undermines the legitimacy of moderation as a neutral process. They are not equally accessible, often in ways that mirror off-platform power structures. However, these sorts of public appeals often provide moderators with otherwise inaccessible or illegible context. Knowledge of a videographers’ portfolio, past citation of a pseudonymous expert, or the jovial nature of #bitcheswhobrunch can all help inform platform decisions. Unfortunately, from the outside, it can be difficult to know whether moderators have caved to public pressure or independently determined that their initial decision was made in error.
This is not exclusively a Twitter problem, though the platform’s default openness may make the issue more visible there. A formal mechanism for the provision of additional context would improve other platforms, as well. Ford Fischer, an independent videographer, was banned from Facebook after posting footage of armed protestors before being reinstated after appealing to his followers on Twitter. He writes:
I wish I had a formal avenue to say, “I cover — not participate in – activism including that of paramilitaries in my capacity as a journalist.” Instead, I had to resort to a viral post on Twitter.
Off-platform calls to action have long been a popular method of informal appeal. When Rose McGowan was locked out of her Twitter account after castigating Ben Affleck for denying knowledge of Harvey Weinstein’s predatory behavior, she took to Instagram to protest the decision. However, like other sorts of informal appeals, off-platform pleas turn on celebrities, placing them beyond the reach of most users. In Fischer’s case, one Facebook employee even contacted him via Twitter to tell him that Facebook staffers had submitted internal appeal requests on his behalf. These may have gotten the job done but, as Fischer explains:
My qualm here is that most people do not have the extraordinary following that I do. When I had a much smaller following, I could have just as easily lost my account without the community backing to speak out.
Platforms could formalize the ad hoc process of collaborative appeals and bring it back on-platform by allowing banned users to tag their followers or those familiar with their case in the appeals process. Tagged users could then choose to submit written testimony or pertinent evidence that might alter moderators’ analysis of the incident or content in question. In many cases, moderators might not need that much additional information. The knowledge that a given user is a journalist merely covering a riot or that a quote deemed hateful comes from the Declaration of Independence is not difficult to convey.
Providing a formal mechanism for collaborative appeals would reduce the perception of unfairness that accompanies platforms’ responsiveness to the informal process, potentially making such appeals more useful to moderators at the same time. It would provide a counterweight to mass bad-faith reporting — a frequent misuse of platform user reporting tools. While moderators have limited resources to examine additional evidence, the mere fact that a given decision receives substantial pushback often indicates that it warrants further review.
User flagging is undoubtedly a useful and necessary feature that helps platforms catch violative content their moderators might otherwise miss; like other social media tools, however, it can be used irresponsibly. Often guided by quote tweets, users report non-violative content en masse merely because they disagree with it or dislike its author. Sometimes, as may have occurred in @MENA_Conflict’s case, this deluge of false reports is enough to spur platform action. While user flagging remains valuable despite this potential for abuse, the effects of its misuse could be curbed by the addition of collaborative appeal features. Given the difficulty of modifying the reporting feature to prevent bad faith flagging without undermining its usefulness, formalizing the emergent norm of meeting brigade with brigade seems like the best way forward.
Obviously, this would not eliminate inequalities between high and low follower accounts, but it would allow small accounts to access some aspects of the informal appeal currently enjoyed by larger ones. It might also enable moderation to be more responsive to context without forcing moderators to pick between competing sets of facts. Determining context, or the correct context in which to view some content, remains one of the most difficult aspects of content moderation. However, there are good reasons to privilege, or at least recognize, the meaning of speech as understood by its most immediate recipients or participants in the original conversation. They are its local, intended audience — those most likely to appreciate its meaning.
Use of nonstandard English vernacular is often deemed offensive by overzealous algorithms, despite universal agreement as to its inoffensive connotations by the original speakers and listeners. Perhaps we could think of it as recognizing the “original public meaning” of speech, accepting that, as in the physical world, the internet’s little platoons often have their own dialects. Like a common law jury of one’s peers, a formal collaborative appeal mechanism could provide an understanding of local norms and conditions likely to be overlooked by external experts.
Users tagged in an ostensibly threatening post could easily be queried to provide additional information, for example, that the post’s author was merely quoting her friend’s lighthearted threat to kill her husband if he removed her plate. This low-hanging fruit – preventing moderation from treating conversations between friends as harassment – might not need a human in the loop at all. Machines may struggle to glean context, but they can accept a “not offensive” input from an apparent victim.
Features that allow users’ friends or participants in a specific conversation to offer testimony would provide benefits for large and small accounts alike. The informal appeal has emerged to solve a particular problem inherent to content moderation at scale and, inegalitarian as it may be, it cannot be wished away. However, by formalizing this redress mechanism, platforms can transform an elite privilege into a tool for everyone.
