South ! \'/ E -- -- W /|\ ' N habari.53189.net The state of News As 2005 begins it has become obvious that News (a world-wide distributed interactive system consisting of "newsgroups" with names which are classified hierarchically by subject) is floundering. A study of News activity shows that since about 2001 there has been a general abandonment of News management by Newsgroup Moderators. In 2004 finding the description of a particular newsgroup has become difficult, and finding the relay address for the moderator of a moderated newsgroup has become almost an impossiblity. Many formerly moderated groups, although supposedly still moderated, have either stopped attempting to manage postings or have quietly left News entirely and opened up as Web based "forums" (the abandoned News group then becomming nothing but a Spam bucket). Why is this? I believe it is due primarily to the way News functions. Moderators are usually volunteers. Although the News user base had been growing steadily, as more people got computers and as more ISPs added News service, the moderators were still able to keep up with demand. But then came the email harvesting robots. These robots managed to gather the email addresses of the moderators and proceeded to fill their inboxes with Spam. This made the job of the moderator very difficult, as his mail load suddenly trebbled. Still, the moderators could have turned the situation around at this point. Moderators could have reconfigured their mailboxes to only accept email sent by News servers, rather than accept mail sent by anyone. This would have meant that individuals would have been restricted to only posting their messages by first routing them through a News server. This would not have been much of an issue, as users usually post back up through the server they connect with to read the newsgroups anyways. The real problem, and why moderators did not take this course of action, was due to the restriction of only accepting mail from News servers. Traditionally News servers have posted to moderators anonymously. Furthermore, the majority of News servers never posted to the moderator anyways. They just added messages to their local copy of the newsgroup as if the moderated group was just another unmoderated newsgroup (this is probably becuase the power base in News seems to be rested in the hands of the server administrators rather than the newsgroup moderators). And this is what I believe is the real heart of the problem... The irresponsability of News Admins. To be fair, this often is not done intentionally. There really is no difinitive manual that is widely available teaching one how to responsibly operate a News server. Most persons administering the server are not full time administrators anyways, just random ISP employees assigned the job of occationally glancing at the logs to see if the server is still running properly. To a typical ISP, News is just a service you turn on and then forget about. Most News administrators learn about administration by reading the manual for their News software. Unfortunately, these manuals are somewhat of a contributor to the problem, as they merely mention how to send messages to moderators annonymously. They do not stress the importance of ONLY sending messages from moderated groups to moderators. Furthermore, these manuals encourage the taking of feeds using groupname wildcards, the posting of messages locally, and the rapid dissemination to other News servers of any messages posted locally. In the past this attitude of take all and give all would have been tolerable. However, Spammers have learned that this is just what they need to get their advertising delivered. In some newsgroups the majority of posts are now nothing but Spam. Server administrators are now complaining about the recent heavy and growing News traffic that is filling up their hard drives and taking longer to relay. But although some of this traffic is due to increased userbase (especially in the "binaries" groups), it is mostly due to Spam. How can Spam be reduced? Lots of ways. Administrators can restrict users from crossposting (with modern "cut&paste" graphical user interfaces, sending a duplicate message to each group is no longer difficult). A crossposting restriction also prevents spammers from entering a non-carried newsgroup via a carried group crosspost. Asministrators can actively black list known spammer user names and IP addresses from entering their systems. Admins can do all this and more, and much of it can be done automatically by filters. But the most effective anti-Spam tool of all is the responsable management of newsgroups. Don't propogate Spam! The "we carry a full feed" target is foolish, as the majority of existant news groups are just Spam buckets and have NO user placed messages in them at all. Don't add newsgroups just because they exist on your feeder. All you are doing is wasting you own precious resources, because your local users are NOT reading these groups. Use groupname wildcards judiciously! Don't ever locally post a message to moderated group! Always send a moderated message up to the moderator WITHOUT posting locally. If the moderator approves the message, it WILL come back to you through a feed. Posting a message locally is an invitation to Spam. The spammer knows that his message will never get through the moderator. However, because you posted it locally, when you send your feed to the next News server (or nowdays, when the other News server "sucks" directly from your local newsgroup), you open an avenue for the Spam to get out and pollute the rest of the News. Even if the other servers were careful about keeping Spam out of moderated groups, your feed will now dump that Spam back into their local newsgroup copies anyways. Can News be saved? Yes, but ONLY if News server administrators take an active role in newsgroup administration, and ONLY if moderators can be convinced to continue to keep their groups on News. Active moderators MUST be proactive and inform the server administrators Who they are, and Where posts should be sent! News administration can also be improved by moving towards a heirarchal model somewhat like FidoNet, where servers are autonomous but part of a network. System administrators should not accept or propogate "Control Cancel" messages. They were never intended for use on a globaly distributed system like News. News admins should never allow crossposting. If a user wants to post in a different group, he must send the message to that group as the primary address. There is no longer any user burden associated with sending separate messages. Break the email link. Newsgroups are public forums intended to be read like a bulletin board. Putting a legitimate email address on these posts is bad for two reasons. First, robots scan newsgroups looking for email addresses so they can send email Spam. Second, having an email address means someone can, and often does, send a reply directly to the poster, bypassing the newsgroup. If other readers are following the thread and are waiting for an answer which may be important to them as well, when the reply is only emailed to the poster, then the other readers will never learn the answer. Forcing the responders to only respond via the newsgroup both benifits others and eliminates all potential of posting spawned email harrassment. An excellent way to salvage News is to avoid full feeds. There are super- servers that hold terrabytes of information for which they charge a fee subscription to access. Unfortunately, these are just black holes. While they are a good source for articles when you are looking for multi-part completion and long retention, they are also indescriminate as to what they store. Many of the newsgroups on these servers have long passed into disuse or are local groups not intended for public distribution. Often a "real" group is scooped up from one peering server, but then that peer server stops carring the group. This group then becomed empty of new legitimate content, but the super-server employee (they don't really "administer") never bothers to look. The orphaned group then becomes nothing but a bucket for local generated Spam. ISPs tend to like these super-servers as a source, because they don't have to do any work on their end (just subscribe and forget). The News network then perpetuates this lazyness through indescriminate newsgroup wildcard relaying. But just because one of the "big guys" does things a certain way, you don't have to. Remember, ultimately its whats good for your local users that really matters. Carefully selecting what newsgroups to host and ignoring the rest is what needs to be done. Go ahead and use the super-servers, but DON'T wildcard. If your users really want a specific group, they WILL ask (so do make sure that your server has an method for accepting user requests). Also be careful as to what and how you relay, so you don't become a distributor for spammers.