Google has started an initiative to devalue comment spam. They have got together with MSN, Yahoo and a bunch of blogging companies and come up with a tag which instructs the search engines to ignore this link for PageRank / link popularity calculations. They are using a "rel=nofollow" attribute on href links to do this. Check http://www.google.com/googleblog/2005/01/preventing-comment-spam.html for more details.
Robin Hislop
Spannerworks
Thanks Robin!
Seems you beat me to it. But some two hours ago, I put this up on the Chongqed news page. I think that the idea is really very good and somebody should have had it a long time ago. I hope that this is quickly implemented in wiki engines, too. – Manni - 2005-01-19 13:47
And here is the Slashdot discussion.
So a wiki engine could add this attribute to all externals links, however I think some people would debate whether this is desirable, because legitimate links then no longer receive leg-up in the google ranks.
I'm not really impressed by google's reaction to this problem. They are the ones with the power. Obviously they need to wield it cautiously, but they could drop these spammers off the rankings completely if they just figured out a reliable way to identify them.
On a similar note I was looking at the UseMod site (one of the most popular wiki engines). I was suprised to see that even the latest version of the usemod software does not include 'NOINDEX' meta tags in the old revisions pages. There is a patch available for this, but it is not included in the default install. So I weighed in on the usemod wiki antispam discussions and lobbied for this to be included. – Halz
You sure have a good point here, Halz. But the really good thing about this is that it can currently be found on lots and lots of news services. If somebody has a wiki engine and says in the release notes that kept pages are not indexed, noone will take notice. But if Slashdot, the Register, etc. are carrying a story about Google doing something, that's a completely different thing. I just hope that the spammers take notice and reevaluate the usefulness of what they are doing. – Manni - 2005-01-19 15:35
I am really excited about this. It isn't perfect, but it will solve a large chunk of the spam problem. Mostly it still concentrating on blogs though. Wikis can use it too, but since the entire wiki is user contributed all links would have a nofollow. But this does give the opourtunity for a wiki to have a whitelist of URLs that don't get the new tag. Much of the problem will continue to be old guestbooks, wiki, and blogs that the author has abandoned. – Joe - 2005-01-19 16:15 UTC
For what it's worth, I wrote about a way to do this over a year ago, and developed a Moin Moin patch that did so. it's a little annoying that nobody was paying any attention :( – Jmason - 2005-01-19 13:35 PST
Your idea got similar results, and there were a lot of other places with similar ideas. URL redirects being the most popular till now. Your solution requires Javascript which some people perfer not to use. And redirects require a server that handles it. Google's version is great, this solution is far better than any other so far. Its always been up to them to fix this, its their PageRank that caused it to be so bad. The problem is far from gone, but this is a big step in ending comment spam. – Joe - 2005-01-20 01:36 UTC
There is one thing that sucks about this: It will hurt chongqed.org's PageRank. Wikipedia is already delivering those nofollow links and I'm pretty sure they they are responsible for a good deal of our page rank. Oh well… – Manni - 2005-01-20 17:22
If nofollow become really succesful and widespread there is no need for chongqed.org anyway. I am sure they did give some benifit to our site, but we have enough links now all over the place that it shouldn't hurt too much. It will be when all wikis start using this method that will be the problem. If all links were set to nofollow then a lot of pages will never be indexed, unless nofollow just means don't give the link any PageRank. – Joe - 2005-01-20 19:53 UTC
That's right. And I sure hope that chongqed.org will be just a part of history soon. But with a good page rank, maybe we could have sold the domain for a good price ;-) – Manni - 2005-01-21 10:36
We know one spammer that would probably love to have it. ;-) – Joe - 2005-01-21 09:54 UTC
Is it ok to keep track of spam on other wikis here? I am thinking about this example: http://www.worldwidewiki.net/wiki/HomePage/diff?a=9693&b=9690
Best, MarkDilley?
Hi Mark!
It should be OK to discuss anything that is related to wiki spam here. I'm not quite sure what it is you want to see here specifically. But it would be cool if you could simply look on this wiki to find out about the latest spammers, there links, and methods. We currently have DiscussSpammers and SpamReport. Feel free to create a new page that might better suit your interests. – Manni - 2005-01-18 09:21
Hi Manni! - thanks for the pointers, I found what I was really looking for, The SpamSubmissionForm?. I wanted a way to centrally report spamming/spammers.
Best, MarkDilley?
Where are all the spammers? It's been nearly a wiki since AntiSpamDan caught the last spam attempt. What happened? – Manni - 2005-01-16 11:01
That does seem odd. I read something about email spam lightening up supposedly due to the tsunami. Other than that, maybe it has something to due with the wiki haveing been read only over the holidays, though I doubt spammers keep track of that. They just find a page to spam in Google. – Joe - 2005-01-16 12:27 UTC
And here's a user denying he inserted the text himself:
Does anyone know the name of the spyware program responsible? Or any other information which might help in targetting filters or detecting this software on users' computers?
– TimStarling - 2005-01-12 02:02 UTC
This is the first I have heard of it, a search on the URL included in that users' spyware assisted spam did not bring up a lot, but I did find this page that describes a similar problem. Another wikipedia page that had the link was East Timor, two spammy edits in a row by 4.26.226.65 back on Dec 31. I also found an example from Jan 8 on c2.
This is a very scary spyware, it can make anyone a spammer. Most likely IE users, I wonder if it affects Mozilla. If not a temporary solution until more can be learned is to ask affected people to use FireFox?. Let us know if you find anything else. We will do some research too. From the non-wiki formatting I am assuming its either a stupid spammer or not actually trying to attack wikis specifically. They could be just attacking any page with a text entry box like a forum or guestbook.
– Joe - 2005-01-12 03:46 UTC
I've done a bit more research since I made that post, the most relevant page I've found so far is this. In that report, the links are inserted before the page is displayed to the user. If we're dealing with the same thing or something similar, we could check for inserted links with a javascript onload event. – TimStarling - 2005-01-12 04:53 UTC
Well, for most all wikis javascript is not allowed anyway and will appear just as text. You could just block any JS code that is not inside a pre or code tag. Sometimes its necessary to post code on wikipedia I am sure (especially the page describing Javascript). But that really won't solve the long term problem. I am not sure that is possible. This spyware is currently trying to get hits I for certain sites, but wiki spammers usually only interested in improving their PageRank. Many of the techniques currently useable to detect spammers won't work if regular wiki users are turned into spammers because of spyware. If they started targeting wikis I suspect it will be far harder to detect the spam. – Joe - 2005-01-12 07:35 UTC
Thanks for reporting this here, Tim, and welcome to the chongqed.org wiki!
I guess what we are seeing here wasn't intended to post stuff on wikis. I guess it's a spyware or addware program that parses incoming html (probably working as a proxy), planting links to certain sites in pages. This proxy program must have replaced the text in the textarea you get when editing a wiki page and when the user submitted his changes, the proxy's changes went along for the ride. – Manni - 2005-01-12 15:18
I suspect your right. I hadn't even thought about it just being an accident that it attacks text boxes. That makes a lot more sense why its not after PageRank but hits. Its only trying to get hits from the user of the spyware affected comptuer. – Joe - 2005-01-12 16:28 UTC
Regarding JavaScript?: sorry, I didn't make myself clear. I mean as a Wikipedia system administrator and MediaWiki developer, I can insert JavaScript? code into the headers of all edit pages. We could send a checksum of the contents of the edit box, and then when the page loads, recalculate the checksum with JavaScript?. If they don't match, the user could be redirected to a page informing them of their problem. There's lots of complications, but maybe a variation of this idea would work.
Regarding motives: it's possible it's accidental, or it may be just a lazy programmer realising that with a few keystrokes, they can spray their name through webmail, blogs and wikis. If spamming wikis was their main goal, they could have installed a bot. – TimStarling - 2005-01-13 12:02
Yup. A bot would have been lot's more effective for such a purpose. Still, it's quite scary. Quite scary to imagine many wikipedia articles infested with that kind of crap, quite scary to imagine lots of unknowing users out there who's machines are infected with that kind of crap, and quite scary to see that wikipedia contributors aren't immune to spyware and the huge gaping security-holes in IE. – Manni - 2005-01-13 15:43