The San Francisco Chronicle deceives its readers through comment-deletion trickery

[UPDATE 1: Software Exec Brags About Crypto-Deletion Feature.]
(Sat. 11-24-07, 12:30pm)

[UPDATE 2: Reader Documents "Graylist" of Banned SFGate Users Who Don't Know They're Banned.]
(Sat. 11-24-07, 2:20pm)

[UPDATE 3: The Scandal Spreads: Other Sites Caught Red-Handed Doing the Same Trick.]
(Sat. 11-24-07, 3:25pm)

[UPDATE 4: S.F. Chronicle admits to deceptive comment-deletion policy, offers bizarre excuse, then lies again.]
(Thurs. 11-29-07, 2:58pm)

[All updates can be found below at the end of this entry]

The San Francisco Chronicle has recently activated a devious system by which it deceives commenters on its website, SFGate.com. Here's how it works:

If you make a comment on an article posted at SFGate, and if the site moderators then subsequently delete your comment for whatever reason, it will only appear as deleted to the other readers. HOWEVER, your comment will NOT appear to be deleted if viewed from your own computer! The Chronicle's goal is to trick deleted commenters into not knowing their comments were in fact deleted. I'll give evidence below showing how they do this.

Why would SFGate do such a thing? Because ever since public input was first allowed at SFGate, many commenters who had their comments deleted would come back onto the comment thread and point out that they had been silenced for ideological reasons -- i.e. they weren't sufficiently "progressive" -- or because they had pointed out ethical lapses at SFGate and the Chronicle. Or any number of other reasons that the Chronicle did not want known. So, to pacify these problematic commenters, the SFGate moderators came up with a very clever and underhanded coding trick to prevent deleted commenters from ever finding out that they had been silenced.

Now, I'm certain that there are plenty of comments on SFGate that indeed merit deletion, and plenty of commenters who say patently offensive things. No question about that, and no one is questioning the Chronicle's right to delete such comments. But there are many other comments that get removed for no apparent reason, except for their political stance, or because they strike too close to home -- pointing out flaws in the article's reporting or writing itself, or ethical or moral misdeeds on the part of the Chronicle editors or management. Deleting comments such as those would be bad enough, but the Chronicle really crossed the line with their new technique of essentially lying to any commenter who has been deleted by not allowing them to even know they were deleted -- so they don't subsequently complain.

The flaw in this new system -- and how I discovered the trickery -- is the "Recommended" rankings under each comment. Readers are permitted to "Recommend" comments they like, and the most popular comments can accumulate dozens of "Recommends." Also, comments near the beginning of any thread often get the most "Recommends." Several times over the last few weeks I noticed something odd: on the first page of comments, people would voice an opinion on an article, then I would make a comment that essentially voiced the same opinion as commenters before and after me; but if I checked back several hours later, or the next day, I would notice that these other commenters would have accrued 20 or 30 or more "Recommends," whereas I would either have 0 Recommends or (if I had recommended my own comment, which many people do) just 1 Recommend.

Why was this happening repeatedly? One explanation is that my comments were terrible, and thus did not earn any "Recommends," but in most cases they were not much different from other comments, and were at least as well-written. But today I discovered by accident the real reason. The night before I had made a comment from my personal computer about this SFGate article. When I checked back this morning, I noticed once again that my comment was the only comment which had only 1 "Recommend." I didn't give it much thought, but later in the day I revisited the same thread from a friend's computer. To my surprise, I discovered that my comment had been deleted, when viewed on this other computer. Then, later, I returned to my own computer, revisited the same thread -- and my comment had mysteriously reappeared, at least from my point of view.

Suspicious, I then took the following steps: I deleted the "cookies" that SFGate installs on users' computers to identify who they are, then logged out of my account, and then revisited the same thread on my own computer. As I suspected, the comment was no longer visible, replaced by the moderator's notation "This comment has been removed by SFGate." Then when I logged back in to my account, and viewed the thread as "me" again -- the comment was once again visible.

I also confirmed this by viewing the comment thread using a different browser (Firefox) which did not have any SFGate cookies installed yet, and on which I had not logged in to my account. Sure enough, the comment appeared as deleted; while exactly simultaneously, using my original logged-in browser (Safari) the comment was not deleted.

In other words, whenever I viewed the comments thread as "myself" (i.e. logged in under my account name, which in this case was "jimjams"), my comment remained visible; but whenever I viewed the comment thread either anonymously (i.e. not logged in) or from a browser with no SFGate cookies or (most importantly) from some other computer, then my comment was gone -- deleted by the moderators.

So the end result is that the only person who can see a deleted comment is the person who originally made that comment. To everyone else in the world -- the comment is gone, deleted, non-existent. And the only conceivable purpose for this is to trick commenters into not knowing their comments had been deleted.

Thus, I issue this call to anyone who has ever suspected that their comment was deleted at SFGate, or who ever was stuck at 0 "Recommends" near the beginning of a comment thread even though you made an excellent or incisive comment: take the steps I describe above, and you almost certainly will have the same experience that I did: your comment will be deleted everywhere except your home logged-in computer.

To provide some evidence of my claims, I have taken the following screenshot which shows exactly what I'm describing; the direct link to this particular comments thread is here. Click on this image to view it full-size:



This screenshot shows two different browser windows open simultaneously: The left side shows how the comments thread appears to me when I'm logged out, having cleared my cookies; and the right side shows how the comments thread appears to me when I'm logged in with cookies turned back on. Notice that my comment (the one by "jimjams") is missing on the left side yet present on the right side; and that all the other comments have a greater number of "Recommends" on the right side, meaning that that window was opened necessarily at a later time -- and yet that's the side with the visible comment, meaning that it must have become visible AFTER having already been deleted.

Please note that this is not a debate over whether or not this one particular comment of mine merited deletion -- I disagreed with the article's author, and called him an "idiot" for completely misrepresenting the issue, which I suppose the moderators felt was too extreme. No, the issue is that they tried to hide the fact of the deletion from me through chicanery -- and I suspect that they pull the same trick on other deleted commenters too, in order to pacify them.

Do you suspect that has ever happened to you on a comments thread at SFGate? If so, try the various steps described above (deleting cookies, logging out, and re-viewing the thread; or viewing it with a different browser, or on a different computer). Readers are invited to post their experiences here as comments on this blog. And I encourage you to post this URL (http://investigatethemedia.blogspot.com/2007/11/san-francisco-chronicle-deceives-its.html) in SFGate comments wherever you think it's appropriate.

-------------

Reader "Afkovach" sends in these two screenshots: the first one shows how his comments are visible when he is logged in to SFGate; and the second one shows that they appear as deleted when he views SFGate after logging out. This confirms my experience.

-------------


[UPDATE 1, Sat., 11-24-07, 12:30pm]: Software Exec Brags About Crypto-Deletion Feature

The PBS blog "MediaShift" recently had an interview with Rich Skrenta, the former CEO of a commenting-forum-software company called Topix, in which he bragged about this capability in his company's product:

There’s a lot of tricks in it. For instance, if you are banned from the forums, you can actually still post, and see your own posts, but other people don’t see them. That’s a neat social trick, because if you know you’ve been banned, most people will work around that. They’ll clear their cookies and work to figure out how to get around the block; but if they don’t know they’ve been banned, and they seem to be able to post, it won’t do any harm to the environment.

Another software company called Prospero supposedly also makes commenting software with this crypto-deletion feature. However, I don't know for sure if SFGate uses either the Topix or Prospero software. (Hat tip: MonkeySon)


[UPDATE 2, Sat., 11-24-07, 2:20pm]: Reader Documents "Graylist" of Banned SFGate Users Who Don't Know They're Banned

ITM commenter "Bricology" has just documented that, at least in some cases, the comment-deletions on SFGate are automated; that all comments from certain users who have been secretly banned from the site are immediately deleted automatically; but that such deletions are not visible to the banned commenter himself. Thus, he never knows that he has been banned.

Bricology took these three consecutive screenshots: click on each one to see full-size versions.





The first picture shows an innocuous comment on a thread that is visible when the commenter is logged in to his SFGate account. The second screenshot shows that the comment has already been deleted just a minute later -- when viewed as a non-SFGate member who is not logged in. (Notice the timestamps at the bottom right of the images.) And then the third picture shows the exact same comment reappeared on SFGate once more shortly afterwards -- after the commenter re-logged in to his account.

Since the comment was non-abusive and entirely innocuous (on purpose, as a test), and since it was deleted immediately, and yet still visible to the owner of the account, this proves that certain users are on a "graylist" of banned users who are not informed they are banned. Kudos to Bricology for discovering this.


[UPDATE 3, Sat., 11-24-07, 3:25pm]: The Scandal Spreads: Other Sites Caught Red-Handed Doing the Same Trick

The DJ Konservo site has just posted screenshots taken from the mainstream blog ThinkProgress which prove that the exact same comment deletion deception is being implemented there as well.

We're also getting reports from readers that newspapers in Canada, Washington state and Houston may also be using the same underhanded software gimmick -- preventing deleted or banned commenters from ever discovering that they have been deleted or banned.


[UPDATE 4, Thurs. 11-29-07, 2:58pm]: S.F. Chronicle admits to deceptive comment-deletion policy, offers bizarre excuse, then lies again.]

In a new posting on Investigate the Media, we discuss an interview given by the Webmaster for SFGate, in which she admits that the Chronicle did indeed crypto-delete "graylisted" commenters, as we claimed, then goes on to offer a bizarre excuse for this behavior that makes little sense, and finally falsely states that they have ended the practice, which in fact they haven't. Go to the new thread to read all the details.