# HG changeset patch # User chris # Date 1246027730 -7200 # Node ID 6fceb66e1ad7e70dbd98679966c7cd512c151f01 # Parent f907866f0e4be6f0d0d13e2d3c4a1207d9cd180a Hooray... sonews/0.5.0 final HG: Enter commit message. Lines beginning with 'HG:' are removed. HG: Remove all lines to abort the collapse operation. diff -r f907866f0e4b -r 6fceb66e1ad7 .hgtags --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/.hgtags Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,2 @@ +42b394eda04ba06126b04e66606ff9ce769652fc oneThreadPerSocket +19130f88c6b80cbcda5626c0a49fb35b28a8e3cb sonews-0.5.0 diff -r f907866f0e4b -r 6fceb66e1ad7 AUTHORS --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/AUTHORS Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,26 @@ +AUTHORS & CREDITS +================= + +As most software applications "sonews" is based on the work +of individuals or projects. These fine people contributing to +the Free Software community are mentioned here: + +sonews News Server +------------------- +Copyright (c) 2009 by Christian Lins + +based partly upon + +Neat NNTP Daemon (n3tpd) +------------------------ +Copyright (c) 2007, 2008 by Christian Lins + +based partly upon + +tnntpd +------ +Copyright (c) 2003 by Dennis Schwerdel + +If you find someone missing here, please contact the project leader! + +Thanks to Sun Microsystems for supporting this project!! diff -r f907866f0e4b -r 6fceb66e1ad7 COPYING --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/COPYING Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN-web/README.Debian --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN-web/README.Debian Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,6 @@ +cync for Debian +--------------- + +cync is still a very early alpha version, so be careful using it. You have been warned! + + -- Jens Mühlenhoff Mon, 11 Aug 2008 17:05:23 +0200 diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN-web/compat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN-web/compat Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +6 diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN-web/control --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN-web/control Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,15 @@ +Source: sonews +Section: web +Priority: optional +Maintainer: Christian Lins +Homepage: http://www.sonews.org/ +Package: sonews-web +Version: 0.5.0-beta1 +Architecture: all +Depends: kitten, libjchart2d-java, sonews +Description: Webinterface for sonews + sonews is a modern Usenet server providing newsgroups via NNTP. + The lightweight servlet server kitten is used to provide an optional + configuration web interface. + This metapackage depends on all required prerequisites to run the + servlet based webinterface of sonews. diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN/README.Debian --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN/README.Debian Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,6 @@ +cync for Debian +--------------- + +cync is still a very early alpha version, so be careful using it. You have been warned! + + -- Jens Mühlenhoff Mon, 11 Aug 2008 17:05:23 +0200 diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN/compat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN/compat Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +6 diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN/conffiles --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN/conffiles Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +/etc/sonews/sonews.conf diff -r f907866f0e4b -r 6fceb66e1ad7 DEBIAN/control --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/DEBIAN/control Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,15 @@ +Source: sonews +Section: news +Priority: optional +Maintainer: Christian Lins +Homepage: http://www.sonews.org/ +Package: sonews +Version: 0.5.0 +Architecture: all +Depends: openjdk-6-jre-headless | openjdk-6-jre | sun-java6-jre | cacao | jamvm, glassfish-mail, libmysql-java +Suggests: kitten, libpg-java, mysql-server, libjchart2d-java +Description: Usenet news server + sonews is a modern Usenet server providing newsgroups via NNTP. + A relational database backend is used to store the news data. + The lightweight servlet server kitten is used to provide an optional + configuration web interface. diff -r f907866f0e4b -r 6fceb66e1ad7 README --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/README Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,27 @@ +sonews README +============= + +Prerequisites: +-------------- + +* Java 6 Runtime or higher +* MySQL v5 or PostgreSQL v8 or higher +* JDBC driver for your database system + + +Installation: +------------- + +* Create a database in your DBMS, e.g. named like 'sonews' +* Create the necessary table structure using the helpers/*.sql file + (you may use the experimental helper application: + java -cp sonews.jar: DatabaseSetup ) +* Customize the settings within the sonews.conf file or add config values to + the 'config' table. +* Invoke 'bin/sonews.sh start' to start the daemon + +Bugs and other Issues: +---------------------- + +Please mail them to christian.lins@fh-osnabrueck.de or better issue them +into the bugtracker at http://bugs.xerxys.info/ . \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 RFC3977 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/RFC3977 Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,6998 @@ + +Network Working Group C. Feather +Request for Comments: 3977 THUS plc +Obsoletes: 977 October 2006 +Updates: 2980 +Category: Standards Track + + + Network News Transfer Protocol (NNTP) + +Status of This Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (2006). + +Abstract + + The Network News Transfer Protocol (NNTP) has been in use in the + Internet for a decade, and remains one of the most popular protocols + (by volume) in use today. This document is a replacement for + RFC 977, and officially updates the protocol specification. It + clarifies some vagueness in RFC 977, includes some new base + functionality, and provides a specific mechanism to add standardized + extensions to NNTP. + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 + 1.1. Author's Note . . . . . . . . . . . . . . . . . . . . . . 4 + 2. Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 3. Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . 6 + 3.1. Commands and Responses . . . . . . . . . . . . . . . . . 6 + 3.1.1. Multi-line Data Blocks . . . . . . . . . . . . . . . . 8 + 3.2. Response Codes . . . . . . . . . . . . . . . . . . . . . 9 + 3.2.1. Generic Response Codes . . . . . . . . . . . . . . . 10 + 3.2.1.1. Examples . . . . . . . . . . . . . . . . . . . . 12 + 3.3. Capabilities and Extensions . . . . . . . . . . . . . . . 14 + 3.3.1. Capability Descriptions . . . . . . . . . . . . . . . 14 + 3.3.2. Standard Capabilities . . . . . . . . . . . . . . . . 15 + 3.3.3. Extensions . . . . . . . . . . . . . . . . . . . . . 16 + 3.3.4. Initial IANA Register . . . . . . . . . . . . . . . . 18 + 3.4. Mandatory and Optional Commands . . . . . . . . . . . . . 20 + + + +Feather Standards Track [Page 1] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + 3.4.1. Reading and Transit Servers . . . . . . . . . . . . . 21 + 3.4.2. Mode Switching . . . . . . . . . . . . . . . . . . . 21 + 3.5. Pipelining . . . . . . . . . . . . . . . . . . . . . . . 22 + 3.5.1. Examples . . . . . . . . . . . . . . . . . . . . . . 23 + 3.6. Articles . . . . . . . . . . . . . . . . . . . . . . . . 24 + 4. The WILDMAT Format . . . . . . . . . . . . . . . . . . . . . 25 + 4.1. Wildmat Syntax . . . . . . . . . . . . . . . . . . . . . 26 + 4.2. Wildmat Semantics . . . . . . . . . . . . . . . . . . . . 26 + 4.3. Extensions . . . . . . . . . . . . . . . . . . . . . . . 27 + 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . 27 + 5. Session Administration Commands . . . . . . . . . . . . . . . 28 + 5.1. Initial Connection . . . . . . . . . . . . . . . . . . . 28 + 5.2. CAPABILITIES . . . . . . . . . . . . . . . . . . . . . . 29 + 5.3. MODE READER . . . . . . . . . . . . . . . . . . . . . . . 32 + 5.4. QUIT . . . . . . . . . . . . . . . . . . . . . . . . . . 34 + 6. Article Posting and Retrieval . . . . . . . . . . . . . . . . 35 + 6.1. Group and Article Selection . . . . . . . . . . . . . . . 36 + 6.1.1. GROUP . . . . . . . . . . . . . . . . . . . . . . . . 36 + 6.1.2. LISTGROUP . . . . . . . . . . . . . . . . . . . . . . 39 + 6.1.3. LAST . . . . . . . . . . . . . . . . . . . . . . . . 42 + 6.1.4. NEXT . . . . . . . . . . . . . . . . . . . . . . . . 44 + 6.2. Retrieval of Articles and Article Sections . . . . . . . 45 + 6.2.1. ARTICLE . . . . . . . . . . . . . . . . . . . . . . . 46 + 6.2.2. HEAD . . . . . . . . . . . . . . . . . . . . . . . . 49 + 6.2.3. BODY . . . . . . . . . . . . . . . . . . . . . . . . 51 + 6.2.4. STAT . . . . . . . . . . . . . . . . . . . . . . . . 53 + 6.3. Article Posting . . . . . . . . . . . . . . . . . . . . . 56 + 6.3.1. POST . . . . . . . . . . . . . . . . . . . . . . . . 56 + 6.3.2. IHAVE . . . . . . . . . . . . . . . . . . . . . . . . 58 + 7. Information Commands . . . . . . . . . . . . . . . . . . . . 61 + 7.1. DATE . . . . . . . . . . . . . . . . . . . . . . . . . . 61 + 7.2. HELP . . . . . . . . . . . . . . . . . . . . . . . . . . 62 + 7.3. NEWGROUPS . . . . . . . . . . . . . . . . . . . . . . . . 63 + 7.4. NEWNEWS . . . . . . . . . . . . . . . . . . . . . . . . . 64 + 7.5. Time . . . . . . . . . . . . . . . . . . . . . . . . . . 65 + 7.5.1. Examples . . . . . . . . . . . . . . . . . . . . . . 66 + 7.6. The LIST Commands . . . . . . . . . . . . . . . . . . . . 66 + 7.6.1. LIST . . . . . . . . . . . . . . . . . . . . . . . . 67 + 7.6.2. Standard LIST Keywords . . . . . . . . . . . . . . . 69 + 7.6.3. LIST ACTIVE . . . . . . . . . . . . . . . . . . . . . 70 + 7.6.4. LIST ACTIVE.TIMES . . . . . . . . . . . . . . . . . . 71 + 7.6.5. LIST DISTRIB.PATS . . . . . . . . . . . . . . . . . . 72 + 7.6.6. LIST NEWSGROUPS . . . . . . . . . . . . . . . . . . . 73 + 8. Article Field Access Commands . . . . . . . . . . . . . . . . 73 + 8.1. Article Metadata . . . . . . . . . . . . . . . . . . . . 74 + 8.1.1. The :bytes Metadata Item . . . . . . . . . . . . . . 74 + 8.1.2. The :lines Metadata Item . . . . . . . . . . . . . . 75 + 8.2. Database Consistency . . . . . . . . . . . . . . . . . . 75 + + + +Feather Standards Track [Page 2] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + 8.3. OVER . . . . . . . . . . . . . . . . . . . . . . . . . . 76 + 8.4. LIST OVERVIEW.FMT . . . . . . . . . . . . . . . . . . . . 81 + 8.5. HDR . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 + 8.6. LIST HEADERS . . . . . . . . . . . . . . . . . . . . . . 87 + 9. Augmented BNF Syntax for NNTP . . . . . . . . . . . . . . . . 90 + 9.1. Introduction . . . . . . . . . . . . . . . . . . . . . . 90 + 9.2. Commands . . . . . . . . . . . . . . . . . . . . . . . . 92 + 9.3. Command Continuation . . . . . . . . . . . . . . . . . . 93 + 9.4. Responses . . . . . . . . . . . . . . . . . . . . . . . . 93 + 9.4.1. Generic Responses . . . . . . . . . . . . . . . . . . 93 + 9.4.2. Initial Response Line Contents . . . . . . . . . . . 94 + 9.4.3. Multi-line Response Contents . . . . . . . . . . . . 94 + 9.5. Capability Lines . . . . . . . . . . . . . . . . . . . . 95 + 9.6. LIST Variants . . . . . . . . . . . . . . . . . . . . . . 96 + 9.7. Articles . . . . . . . . . . . . . . . . . . . . . . . . 97 + 9.8. General Non-terminals . . . . . . . . . . . . . . . . . . 97 + 9.9. Extensions and Validation . . . . . . . . . . . . . . . . 99 + 10. Internationalisation Considerations . . . . . . . . . . . . .100 + 10.1. Introduction and Historical Situation . . . . . . . . . .100 + 10.2. This Specification . . . . . . . . . . . . . . . . . . .101 + 10.3. Outstanding Issues . . . . . . . . . . . . . . . . . . .102 + 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .103 + 12. Security Considerations . . . . . . . . . . . . . . . . . . .103 + 12.1. Personal and Proprietary Information . . . . . . . . . .104 + 12.2. Abuse of Server Log Information . . . . . . . . . . . . .104 + 12.3. Weak Authentication and Access Control . . . . . . . . .104 + 12.4. DNS Spoofing . . . . . . . . . . . . . . . . . . . . . .104 + 12.5. UTF-8 Issues . . . . . . . . . . . . . . . . . . . . . .105 + 12.6. Caching of Capability Lists . . . . . . . . . . . . . . .106 + 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . .107 + 14. References . . . . . . . . . . . . . . . . . . . . . . . . .110 + 14.1. Normative References . . . . . . . . . . . . . . . . . .110 + 14.2. Informative References . . . . . . . . . . . . . . . . .110 + A. Interaction with Other Specifications . . . . . . . . . . . .112 + A.1. Header Folding . . . . . . . . . . . . . . . . . . . . .112 + A.2. Message-IDs . . . . . . . . . . . . . . . . . . . . . . .112 + A.3. Article Posting . . . . . . . . . . . . . . . . . . . . .114 + B. Summary of Commands . . . . . . . . . . . . . . . . . . . . .115 + C. Summary of Response Codes . . . . . . . . . . . . . . . . . .117 + D. Changes from RFC 977 . . . . . . . . . . . . . . . . . . . .121 + +1. Introduction + + This document specifies the Network News Transfer Protocol (NNTP), + which is used for the distribution, inquiry, retrieval, and posting + of Netnews articles using a reliable stream-based mechanism. For + news-reading clients, NNTP enables retrieval of news articles that + + + + +Feather Standards Track [Page 3] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + are stored in a central database, giving subscribers the ability to + select only those articles they wish to read. + + The Netnews model provides for indexing, cross-referencing, and + expiration of aged messages. NNTP is designed for efficient + transmission of Netnews articles over a reliable full duplex + communication channel. + + Although the protocol specification in this document is largely + compatible with the version specified in RFC 977 [RFC977], a number + of changes are summarised in Appendix D. In particular: + + o the default character set is changed from US-ASCII [ANSI1986] to + UTF-8 [RFC3629] (note that US-ASCII is a subset of UTF-8); + + o a number of commands that were optional in RFC 977 or that have + been taken from RFC 2980 [RFC2980] are now mandatory; and + + o a CAPABILITIES command has been added to allow clients to + determine what functionality is available from a server. + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in RFC 2119 [RFC2119]. + + An implementation is not compliant if it fails to satisfy one or more + of the MUST requirements for this protocol. An implementation that + satisfies all the MUST and all the SHOULD requirements for its + protocols is said to be "unconditionally compliant"; one that + satisfies all the MUST requirements but not all the SHOULD + requirements for NNTP is said to be "conditionally compliant". + + For the remainder of this document, the terms "client" and "client + host" refer to a host making use of the NNTP service, while the terms + "server" and "server host" refer to a host that offers the NNTP + service. + +1.1. Author's Note + + This document is written in XML using an NNTP-specific DTD. Custom + software is used to convert this to RFC 2629 [RFC2629] format, and + then the public "xml2rfc" package to further reduce this to text, + nroff source, and HTML. + + No perl was used in producing this document. + + + + + + +Feather Standards Track [Page 4] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +2. Notation + + The following notational conventions are used in this document. + + UPPERCASE indicates literal text to be included in the + command. + + lowercase indicates a token described elsewhere. + + [brackets] indicate that the enclosed material is optional. + + elliptical indicates that the argument may be repeated any + ... marks number of times (it must occur at least once). + + vertical|bar indicates a choice of two mutually exclusive + arguments (exactly one must be provided). + + The name "message-id" for a command or response argument indicates + that it is the message-id of an article as described in Section 3.6, + including the angle brackets. + + The name "wildmat" for an argument indicates that it is a wildmat as + defined in Section 4. If the argument does not meet the requirements + of that section (for example, if it does not fit the grammar of + Section 4.1), the NNTP server MAY place some interpretation on it + (not specified by this document) or otherwise MUST treat it as a + syntax error. + + Responses for each command will be described in tables listing the + required format of a response followed by the meaning that should be + ascribed to that response. + + The terms "NUL", "TAB", "LF", "CR, and "space" refer to the octets + %x00, %x09, %x0A, %x0D, and %x20, respectively (that is, the octets + with those codes in US-ASCII [ANSI1986] and thus in UTF-8 [RFC3629]). + The term "CRLF" or "CRLF pair" means the sequence CR immediately + followed by LF (that is, %x0D.0A). A "printable US-ASCII character" + is an octet in the range %x21-7E. Quoted characters refer to the + octets with those codes in US-ASCII (so "." and "<" refer to %x2E and + %x3C) and will always be printable US-ASCII characters; similarly, + "digit" refers to the octets %x30-39. + + A "keyword" MUST consist only of US-ASCII letters, digits, and the + characters dot (".") and dash ("-") and MUST begin with a letter. + Keywords MUST be at least three characters in length. + + + + + + +Feather Standards Track [Page 5] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Examples in this document are not normative but serve to illustrate + usages, arguments, and responses. In the examples, a "[C]" will be + used to represent the client host and an "[S]" will be used to + represent the server host. Most of the examples do not rely on a + particular server state. In some cases, however, they do assume that + the currently selected newsgroup (see the GROUP command, + Section 6.1.1) is invalid; when so, this is indicated at the start of + the example. Examples may use commands or other keywords not defined + in this specification (such as an XENCRYPT command). These will be + used to illustrate some point and do not imply that any such command + is defined elsewhere or needs to exist in any particular + implementation. + + Terms that might be read as specifying details of a client or server + implementation, such as "database", are used simply to ease + description. Provided that implementations conform to the protocol + and format specifications in this document, no specific technique is + mandated. + +3. Basic Concepts + +3.1. Commands and Responses + + NNTP operates over any reliable bi-directional 8-bit-wide data stream + channel. When the connection is established, the NNTP server host + MUST send a greeting. The client host and server host then exchange + commands and responses (respectively) until the connection is closed + or aborted. If the connection used is TCP, then the server host + starts the NNTP service by listening on a TCP port. When a client + host wishes to make use of the service, it MUST establish a TCP + connection with the server host by connecting to that host on the + same port on which the server is listening. + + The character set for all NNTP commands is UTF-8 [RFC3629]. Commands + in NNTP MUST consist of a keyword, which MAY be followed by one or + more arguments. A CRLF pair MUST terminate all commands. Multiple + commands MUST NOT be on the same line. Unless otherwise noted + elsewhere in this document, arguments SHOULD consist of printable US- + ASCII characters. Keywords and arguments MUST each be separated by + one or more space or TAB characters. Command lines MUST NOT exceed + 512 octets, which includes the terminating CRLF pair. The arguments + MUST NOT exceed 497 octets. A server MAY relax these limits for + commands defined in an extension. + + Where this specification permits UTF-8 characters outside the range + of U+0000 to U+007F, implementations MUST NOT use the Byte Order Mark + (U+FEFF, encoding %xEF.BB.BF) and MUST use the Word Joiner (U+2060, + encoding %xE2.91.A0) for the meaning Zero Width No-Break Space in + + + +Feather Standards Track [Page 6] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + command lines and the initial lines of responses. Implementations + SHOULD apply these same principles throughout. + + The term "character" means a single Unicode code point. + Implementations are not required to carry out Unicode normalisation. + Thus, U+0084 (A-dieresis) is one character, while U+0041 U+0308 (A + composed with dieresis) is two; the two need not be treated as + equivalent. + + Commands may have variants; if so, they use a second keyword + immediately after the first to indicate which variant is required. + The only such commands in this specification are LIST and MODE. Note + that such variants are sometimes referred to as if they were commands + in their own right: "the LIST ACTIVE" command should be read as + shorthand for "the ACTIVE variant of the LIST command". + + Keywords are case insensitive; the case of keywords for commands MUST + be ignored by the server. Command and response arguments are case or + language specific only when stated, either in this document or in + other relevant specifications. + + In some cases, a command involves more data than just a single line. + The further data may be sent either immediately after the command + line (there are no instances of this in this specification, but there + are in extensions such as [NNTP-STREAM]) or following a request from + the server (indicated by a 3xx response). + + Each response MUST start with a three-digit response code that is + sufficient to distinguish all responses. Certain valid responses are + defined to be multi-line; for all others, the response is contained + in a single line. The initial line of the response MUST NOT exceed + 512 octets, which includes the response code and the terminating CRLF + pair; an extension MAY specify a greater maximum for commands that it + defines, but not for any other command. Single-line responses + consist of an initial line only. Multi-line responses consist of an + initial line followed by a multi-line data block. + + An NNTP server MAY have an inactivity autologout timer. Such a timer + SHOULD be of at least three minutes' duration, with the exception + that there MAY be a shorter limit on how long the server is willing + to wait for the first command from the client. The receipt of any + command from the client during the timer interval SHOULD suffice to + reset the autologout timer. Similarly, the receipt of any + significant amount of data from a client that is sending a multi-line + data block (such as during a POST or IHAVE command) SHOULD suffice to + reset the autologout timer. When the timer expires, the server + SHOULD close the connection without sending any response to the + client. + + + +Feather Standards Track [Page 7] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +3.1.1. Multi-line Data Blocks + + A multi-line data block is used in certain commands and responses. + It MUST adhere to the following rules: + + 1. The block consists of a sequence of zero or more "lines", each + being a stream of octets ending with a CRLF pair. Apart from + those line endings, the stream MUST NOT include the octets NUL, + LF, or CR. + + 2. In a multi-line response, the block immediately follows the CRLF + at the end of the initial line of the response. When used in any + other context, the specific command will define when the block is + sent. + + 3. If any line of the data block begins with the "termination octet" + ("." or %x2E), that line MUST be "dot-stuffed" by prepending an + additional termination octet to that line of the block. + + 4. The lines of the block MUST be followed by a terminating line + consisting of a single termination octet followed by a CRLF pair + in the normal way. Thus, unless it is empty, a multi-line block + is always terminated with the five octets CRLF "." CRLF + (%x0D.0A.2E.0D.0A). + + 5. When a multi-line block is interpreted, the "dot-stuffing" MUST + be undone; i.e., the recipient MUST ensure that, in any line + beginning with the termination octet followed by octets other + than a CRLF pair, that initial termination octet is disregarded. + + 6. Likewise, the terminating line ("." CRLF or %x2E.0D.0A) MUST NOT + be considered part of the multi-line block; i.e., the recipient + MUST ensure that any line beginning with the termination octet + followed immediately by a CRLF pair is disregarded. (The first + CRLF pair of the terminating CRLF "." CRLF of a non-empty block + is, of course, part of the last line of the block.) + + Note that texts using an encoding (such as UTF-16 or UTF-32) that may + contain the octets NUL, LF, or CR other than a CRLF pair cannot be + reliably conveyed in the above format (that is, they violate the MUST + requirement above). However, except when stated otherwise, this + specification does not require the content to be UTF-8, and therefore + (subject to that same requirement) it MAY include octets above and + below 128 mixed arbitrarily. + + This document does not place any limit on the length of a line in a + multi-line block. However, the standards that define the format of + articles may do so. + + + +Feather Standards Track [Page 8] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +3.2. Response Codes + + Each response MUST begin with a three-digit status indicator. These + are status reports from the server and indicate the response to the + last command received from the client. + + The first digit of the response broadly indicates the success, + failure, or progress of the previous command: + + 1xx - Informative message + 2xx - Command completed OK + 3xx - Command OK so far; send the rest of it + 4xx - Command was syntactically correct but failed for some reason + 5xx - Command unknown, unsupported, unavailable, or syntax error + + The next digit in the code indicates the function response category: + + x0x - Connection, setup, and miscellaneous messages + x1x - Newsgroup selection + x2x - Article selection + x3x - Distribution functions + x4x - Posting + x8x - Reserved for authentication and privacy extensions + x9x - Reserved for private use (non-standard extensions) + + Certain responses contain arguments such as numbers and names in + addition to the status indicator. In those cases, to simplify + interpretation by the client, the number and type of such arguments + is fixed for each response code, as is whether the code is + single-line or multi-line. Any extension MUST follow this principle + as well. Note that, for historical reasons, the 211 response code is + an exception to this in that the response may be single-line or + multi-line depending on the command (GROUP or LISTGROUP) that + generated it. In all other cases, the client MUST only use the + status indicator itself to determine the nature of the response. The + exact response codes that can be returned by any given command are + detailed in the description of that command. + + Arguments MUST be separated from the numeric status indicator and + from each other by a single space. All numeric arguments MUST be in + base 10 (decimal) format and MAY have leading zeros. String + arguments MUST contain at least one character and MUST NOT contain + TAB, LF, CR, or space. The server MAY add any text after the + response code or last argument, as appropriate, and the client MUST + NOT make decisions based on this text. Such text MUST be separated + from the numeric status indicator or the last argument by at least + one space. + + + + +Feather Standards Track [Page 9] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The server MUST respond to any command with the appropriate generic + response (given in Section 3.2.1) if it represents the situation. + Otherwise, each recognized command MUST return one of the response + codes specifically listed in its description or in an extension. A + server MAY provide extensions to this specification, including new + commands, new variants or features of existing commands, and other + ways of changing the internal state of the server. However, the + server MUST NOT produce any other responses to a client that does not + invoke any of the additional features. (Therefore, a client that + restricts itself to this specification will only receive the + responses that are listed.) + + If a client receives an unexpected response, it SHOULD use the first + digit of the response to determine the result. For example, an + unexpected 2xx should be taken as success, and an unexpected 4xx or + 5xx as failure. + + Response codes not specified in this document MAY be used for any + installation-specific additional commands also not specified. These + SHOULD be chosen to fit the pattern of x9x specified above. + + Neither this document nor any registered extension (see + Section 3.3.3) will specify any response codes of the x9x pattern. + (Implementers of extensions are accordingly cautioned not to use such + responses for extensions that may subsequently be submitted for + registration.) + +3.2.1. Generic Response Codes + + The server MUST respond to any command with the appropriate one of + the following generic responses if it represents the situation. + + If the command is not recognized, or if it is an optional command + that is not implemented by the server, the response code 500 MUST be + returned. + + If there is a syntax error in the arguments of a recognized command, + including the case where more arguments are provided than the command + specifies or the command line is longer than the server accepts, the + response code 501 MUST be returned. The line MUST NOT be truncated + or split and then interpreted. Note that where a command has + variants depending on a second keyword (e.g., LIST ACTIVE and LIST + NEWSGROUPS), 501 MUST be used when the base command is implemented + but the requested variant is not, and 500 MUST be used only when the + base command itself is not implemented. + + If an argument is required to be a base64-encoded string [RFC4648] + (there are no such arguments in this specification, but there may be + + + +Feather Standards Track [Page 10] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + in extensions) and is not validly encoded, the response code 504 MUST + be returned. + + If the server experiences an internal fault or problem that means it + is unable to carry out the command (for example, a necessary file is + missing or a necessary service could not be contacted), the response + code 403 MUST be returned. If the server recognizes the command but + does not provide an optional feature (for example, because it does + not store the required information), or if it only handles a subset + of legitimate cases (see the HDR command, Section 8.5, for an + example), the response code 503 MUST be returned. + + If the client is not authorized to use the specified facility when + the server is in its current state, then the appropriate one of the + following response codes MUST be used. + + 502: It is necessary to terminate the connection and to start a new + one with the appropriate authority before the command can be used. + Historically, some mode-switching servers (see Section 3.4.1) used + this response to indicate that this command will become available + after the MODE READER command (Section 5.3) is used, but this + usage does not conform to this specification and MUST NOT be used. + Note that the server MUST NOT close the connection immediately + after a 502 response except at the initial connection + (Section 5.1) and with the MODE READER command. + + 480: The client must authenticate itself to the server (that is, it + must provide information as to the identity of the client) before + the facility can be used on this connection. This will involve + the use of an authentication extension such as [NNTP-AUTH]. + + 483: The client must negotiate appropriate privacy protection on the + connection. This will involve the use of a privacy extension such + as [NNTP-TLS]. + + 401: The client must change the state of the connection in some other + manner. The first argument of the response MUST be the capability + label (see Section 5.2) of the facility that provides the + necessary mechanism (usually an extension, which may be a private + extension). The server MUST NOT use this response code except as + specified by the definition of the capability in question. + + If the server has to terminate the connection for some reason, it + MUST give a 400 response code to the next command and then + immediately close the connection. Following a 400 response, clients + SHOULD NOT simply reconnect immediately and retry the same actions. + Rather, a client SHOULD either use an exponentially increasing delay + between retries (e.g., double the waiting time after each 400 + + + +Feather Standards Track [Page 11] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + response) or present any associated text to the user for them to + decide whether and when to retry. + + The client MUST be prepared to receive any of these responses for any + command (except, of course, that the server MUST NOT generate a 500 + response code for mandatory commands). + +3.2.1.1. Examples + + Example of an unknown command: + + [C] MAIL + [S] 500 Unknown command + + Example of an unsupported command: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] NEWNEWS + [S] LIST ACTIVE NEWSGROUPS + [S] . + [C] OVER + [S] 500 Unknown command + + Example of an unsupported variant: + + [C] MODE POSTER + [S] 501 Unknown MODE option + + Example of a syntax error: + + [C] ARTICLE a.message.id@no.angle.brackets + [S] 501 Syntax error + + Example of an overlong command line: + + [C] HEAD 53 54 55 + [S] 501 Too many arguments + + Example of a bad wildmat: + + [C] LIST ACTIVE u[ks].* + [S] 501 Syntax error + + + + + + +Feather Standards Track [Page 12] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of a base64-encoding error (the second argument is meant to + be base64 encoded): + + [C] XENCRYPT RSA abcd=efg + [S] 504 Base64 encoding error + + Example of an attempt to access a facility not available to this + connection: + + [C] MODE READER + [S] 200 Reader mode, posting permitted + [C] IHAVE + [S] 500 Permission denied + + Example of an attempt to access a facility requiring authentication: + + [C] GROUP secret.group + [S] 480 Permission denied + + Example of a successful attempt following such authentication: + + [C] XSECRET fred flintstone + [S] 290 Password for fred accepted + [C] GROUP secret.group + [S] 211 5 1 20 secret.group selected + + Example of an attempt to access a facility requiring privacy: + + [C] GROUP secret.group + [S] 483 Secure connection required + [C] XENCRYPT + [Client and server negotiate encryption on the link] + [S] 283 Encrypted link established + [C] GROUP secret.group + [S] 211 5 1 20 secret.group selected + + Example of a need to change mode before a facility is used: + + [C] GROUP binary.group + [S] 401 XHOST Not on this virtual host + [C] XHOST binary.news.example.org + [S] 290 binary.news.example.org virtual host selected + [C] GROUP binary.group + [S] 211 5 1 77 binary.group selected + + + + + + + +Feather Standards Track [Page 13] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of a temporary failure: + + [C] GROUP archive.local + [S] 403 Archive server temporarily offline + + Example of the server needing to close down immediately: + + [C] ARTICLE 123 + [S] 400 Power supply failed, running on UPS + [Server closes connection.] + +3.3. Capabilities and Extensions + + Not all NNTP servers provide exactly the same facilities, both + because this specification allows variation and because servers may + provide extensions. A set of facilities that are related are called + a "capability". This specification provides a way to determine what + capabilities are available, includes a list of standard capabilities, + and includes a mechanism (the extension mechanism) for defining new + capabilities. + +3.3.1. Capability Descriptions + + A client can determine the available capabilities of the server by + using the CAPABILITIES command (Section 5.2). This returns a + capability list, which is a list of capability lines. Each line + describes one available capability. + + Each capability line consists of one or more tokens, which MUST be + separated by one or more space or TAB characters. A token is a + string of 1 or more printable UTF-8 characters (that is, either + printable US-ASCII characters or any UTF-8 sequence outside the US- + ASCII range, but not space or TAB). Unless stated otherwise, tokens + are case insensitive. Each capability line consists of the + following: + + o The capability label, which is a keyword indicating the + capability. A capability label may be defined by this + specification or a successor, or by an extension. + + o The label is then followed by zero or more tokens, which are + arguments of the capability. The form and meaning of these tokens + is specific to each capability. + + The server MUST ensure that the capability list accurately reflects + the capabilities (including extensions) currently available. If a + capability is only available with the server in a certain state (for + example, only after authentication), the list MUST only include the + + + +Feather Standards Track [Page 14] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + capability label when the server is in that state. Similarly, if + only some of the commands in an extension will be available, or if + the behaviour of the extension will change in some other manner, + according to the state of the server, this MUST be indicated by + different arguments in the capability line. + + Note that a capability line can only begin with a letter. Lines + beginning with other characters are reserved for future versions of + this specification. In order to interoperate with such versions, + clients MUST be prepared to receive lines beginning with other + characters and MUST ignore any they do not understand. + +3.3.2. Standard Capabilities + + The following capabilities are defined by this specification. + + VERSION + This capability MUST be advertised by all servers and MUST be the + first capability in the capability list; it indicates the + version(s) of NNTP that the server supports. There must be at + least one argument; each argument is a decimal number and MUST NOT + have a leading zero. Version numbers are assigned only in RFCs + that update or replace this specification; servers MUST NOT create + their own version numbers. + + The version number of this specification is 2. + + READER + This capability indicates that the server implements the various + commands useful for reading clients. + + IHAVE + This capability indicates that the server implements the IHAVE + command. + + POST + This capability indicates that the server implements the POST + command. + + NEWNEWS + This capability indicates that the server implements the NEWNEWS + command. + + HDR + This capability indicates that the server implements the header + access commands (HDR and LIST HEADERS). + + + + + +Feather Standards Track [Page 15] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + OVER + This capability indicates that the server implements the overview + access commands (OVER and LIST OVERVIEW.FMT). If and only if the + server supports the message-id form of the OVER command, there + must be a single argument MSGID. + + LIST + This capability indicates that the server implements at least one + variant of the LIST command. There MUST be one argument for each + variant of the LIST command supported by the server, giving the + keyword for that variant. + + IMPLEMENTATION + This capability MAY be provided by a server. If so, the arguments + SHOULD be used to provide information such as the server software + name and version number. The client MUST NOT use this line to + determine capabilities of the server. (While servers often + provide this information in the initial greeting, clients need to + guess whether this is the case; this capability makes it clear + what the information is.) + + MODE-READER + This capability indicates that the server is mode-switching + (Section 3.4.2) and that the MODE READER command needs to be used + to enable the READER capability. + +3.3.3. Extensions + + Although NNTP is widely and robustly deployed, some parts of the + Internet community might wish to extend the NNTP service. It must be + emphasized that any extension to NNTP should not be considered + lightly. NNTP's strength comes primarily from its simplicity. + Experience with many protocols has shown that: + + Protocols with few options tend towards ubiquity, whilst protocols + with many options tend towards obscurity. + + This means that each and every extension, regardless of its benefits, + must be carefully scrutinized with respect to its implementation, + deployment, and interoperability costs. In many cases, the cost of + extending the NNTP service will likely outweigh the benefit. + + An extension is a package of associated facilities, often but not + always including one or more new commands. Each extension MUST + define at least one new capability label (this will often, but need + not, be the name of one of these new commands). While any additional + capability information can normally be specified using arguments to + + + + +Feather Standards Track [Page 16] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + that label, an extension MAY define more than one capability label. + However, this SHOULD be limited to exceptional circumstances. + + An extension is either a private extension, or its capabilities are + included in the IANA registry of capabilities (see Section 3.3.4) and + it is defined in an RFC (in which case it is a "registered + extension"). Such RFCs either must be on the standards track or must + define an IESG-approved experimental protocol. + + The definition of an extension must include the following: + + o a descriptive name for the extension. + + o the capability label or labels defined by the extension (the + capability label of a registered extension MUST NOT begin with + "X"). + + o The syntax, values, and meanings of any arguments for each + capability label defined by the extension. + + o Any new NNTP commands associated with the extension (the names of + commands associated with registered extensions MUST NOT begin with + "X"). + + o The syntax and possible values of arguments associated with the + new NNTP commands. + + o The response codes and possible values of arguments for the + responses of the new NNTP commands. + + o Any new arguments the extension associates with any other + pre-existing NNTP commands. + + o Any increase in the maximum length of commands and initial + response lines over the value specified in this document. + + o A specific statement about the effect on pipelining that this + extension may have (if any). + + o A specific statement about the circumstances when use of this + extension can alter the contents of the capabilities list (other + than the new capability labels it defines). + + o A specific statement about the circumstances under which the + extension can cause any pre-existing command to produce a 401, + 480, or 483 response. + + + + + +Feather Standards Track [Page 17] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + o A description of how the use of MODE READER on a mode-switching + server interacts with the extension. + + o A description of how support for the extension affects the + behaviour of a server and NNTP client in any other manner not + outlined above. + + o Formal syntax as described in Section 9.9. + + A private extension MAY or MAY NOT be included in the capabilities + list. If it is, the capability label MUST begin with "X". A server + MAY provide additional keywords (for new commands and also for new + variants of existing commands) as part of a private extension. To + avoid the risk of a clash with a future registered extension, these + keywords SHOULD begin with "X". + + If the server advertises a capability defined by a registered + extension, it MUST implement the extension so as to fully conform + with the specification (for example, it MUST implement all the + commands that the extension describes as mandatory). If it does not + implement the extension as specified, it MUST NOT list the extension + in the capabilities list under its registered name. In that case, it + MAY, but SHOULD NOT, provide a private extension (not listed, or + listed with a different name) that implements part of the extension + or implements the commands of the extension with a different meaning. + + A server MUST NOT send different response codes to basic NNTP + commands documented here or to commands documented in registered + extensions in response to the availability or use of a private + extension. + +3.3.4. Initial IANA Register + + IANA will maintain a registry of NNTP capability labels. All + capability labels in the registry MUST be keywords and MUST NOT begin + with X. + + + + + + + + + + + + + + + +Feather Standards Track [Page 18] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The initial content of the registry consists of these entries: + + +-------------------+--------------------------+--------------------+ + | Label | Meaning | Definition | + +-------------------+--------------------------+--------------------+ + | AUTHINFO | Authentication | [NNTP-AUTH] | + | | | | + | HDR | Batched header retrieval | Section 3.3.2, | + | | | Section 8.5, and | + | | | Section 8.6 | + | | | | + | IHAVE | IHAVE command available | Section 3.3.2 and | + | | | Section 6.3.2 | + | | | | + | IMPLEMENTATION | Server | Section 3.3.2 | + | | implementation-specific | | + | | information | | + | | | | + | LIST | LIST command variants | Section 3.3.2 and | + | | | Section 7.6.1 | + | | | | + | MODE-READER | Mode-switching server | Section 3.4.2 | + | | and MODE READER command | | + | | available | | + | | | | + | NEWNEWS | NEWNEWS command | Section 3.3.2 and | + | | available | Section 7.4 | + | | | | + | OVER | Overview support | Section 3.3.2, | + | | | Section 8.3, and | + | | | Section 8.4 | + | | | | + | POST | POST command available | Section 3.3.2 and | + | | | Section 6.3.1 | + | | | | + | READER | Reader commands | Section 3.3.2 | + | | available | | + | | | | + | SASL | Supported SASL | [NNTP-AUTH] | + | | mechanisms | | + | | | | + | STARTTLS | Transport layer security | [NNTP-TLS] | + | | | | + | STREAMING | Streaming feeds | [NNTP-STREAM] | + | | | | + | VERSION | Supported NNTP versions | Section 3.3.2 | + +-------------------+--------------------------+--------------------+ + + + + +Feather Standards Track [Page 19] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +3.4. Mandatory and Optional Commands + + For a number of reasons, not all the commands in this specification + are mandatory. However, it is equally undesirable for every command + to be optional, since this means that a client will have no idea what + facilities are available. Therefore, as a compromise, some of the + commands in this specification are mandatory (they must be supported + by all servers) while the remainder are not. The latter are then + subdivided into bundles, each indicated by a single capability label. + + o If the label is included in the capability list returned by the + server, the server MUST support all commands in that bundle. + + o If the label is not included, the server MAY support none or some + of the commands but SHOULD NOT support all of them. In general, + there will be no way for a client to determine which commands are + supported without trying them. + + The bundles have been chosen to provide useful functionality, and + therefore server authors are discouraged from implementing only part + of a bundle. + + The description of each command will either indicate that it is + mandatory, or will give, using the term "indicating capability", the + capability label indicating whether the bundle including this command + is available. + + Where a server does not implement a command, it MUST always generate + a 500 generic response code (or a 501 generic response code in the + case of a variant of a command depending on a second keyword where + the base command is recognised). Otherwise, the command MUST be + fully implemented as specified; a server MUST NOT only partially + implement any of the commands in this specification. (Client authors + should note that some servers not conforming to this specification + will return a 502 generic response code to some commands that are not + implemented.) + + Note: some commands have cases that require other commands to be used + first. If the former command is implemented but the latter is not, + the former MUST still generate the relevant specific response code. + For example, if ARTICLE (Section 6.2.1) is implemented but GROUP + (Section 6.1.1) is not, the correct response to "ARTICLE 1234" + remains 412. + + + + + + + + +Feather Standards Track [Page 20] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +3.4.1. Reading and Transit Servers + + NNTP is traditionally used in two different ways. The first use is + "reading", where the client fetches articles from a large store + maintained by the server for immediate or later presentation to a + user and sends articles created by that user back to the server (an + action called "posting") to be stored and distributed to other stores + and users. The second use is for the bulk transfer of articles from + one store to another. Since the hosts making this transfer tend to + be peers in a network that transmit articles among one another, and + not end-user systems, this process is called "peering" or "transit". + (Even so, one host is still the client and the other is the server). + + In practice, these two uses are so different that some server + implementations are optimised for reading or for transit and, as a + result, do not offer the other facility or only offer limited + features. Other implementations are more general and offer both. + This specification allows for this by bundling the relevant commands + accordingly: the IHAVE command is designed for transit, while the + commands indicated by the READER capability are designed for reading + clients. + + Except as an effect of the MODE READER command (Section 5.3) on a + mode-switching server, once a server advertises either or both of the + IHAVE or READER capabilities, it MUST continue to advertise them for + the entire session. + + A server MAY provide different modes of behaviour (transit, reader, + or a combination) to different client connections and MAY use + external information, such as the IP address of the client, to + determine which mode to provide to any given connection. + + The official TCP port for the NNTP service is 119. However, if a + host wishes to offer separate servers for transit and reading + clients, port 433 SHOULD be used for the transit server and 119 for + the reading server. + +3.4.2. Mode Switching + + An implementation MAY, but SHOULD NOT, provide both transit and + reader facilities on the same server but require the client to select + which it wishes to use. Such an arrangement is called a + "mode-switching" server. + + + + + + + + +Feather Standards Track [Page 21] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + A mode-switching server has two modes: + + o Transit mode, which applies after the initial connection. + + * It MUST advertise the MODE-READER capability. + + * It MUST NOT advertise the READER capability. + + However, the server MAY cease to advertise the MODE-READER + capability after the client uses any command except CAPABILITIES. + + o Reading mode, after a successful MODE READER command (see Section + 5.3). + + * It MUST NOT advertise the MODE-READER capability. + + * It MUST advertise the READER capability. + + * It MAY NOT advertise the IHAVE capability, even if it was + advertising it in transit mode. + + A client SHOULD only issue a MODE READER command to a server if it is + advertising the MODE-READER capability. If the server does not + support CAPABILITIES (and therefore does not conform to this + specification), the client MAY use the following heuristic: + + o If the client wishes to use any "reader" commands, it SHOULD use + the MODE READER command immediately after the initial connection. + + o Otherwise, it SHOULD NOT use the MODE READER command. + + In each case, it should be prepared for some commands to be + unavailable that would have been available if it had made the other + choice. + +3.5. Pipelining + + NNTP is designed to operate over a reliable bi-directional + connection, such as TCP. Therefore, if a command does not depend on + the response to the previous one, it should not matter if it is sent + before that response is received. Doing this is called "pipelining". + However, certain server implementations throw away all text received + from the client following certain commands before sending their + response. If this happens, pipelining will be affected because one + or more commands will have been ignored or misinterpreted, and the + client will be matching the wrong responses to each command. Since + there are significant benefits to pipelining, but also circumstances + where it is reasonable or common for servers to behave in the above + + + +Feather Standards Track [Page 22] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + manner, this document puts certain requirements on both clients and + servers. + + Except where stated otherwise, a client MAY use pipelining. That is, + it may send a command before receiving the response for the previous + command. The server MUST allow pipelining and MUST NOT throw away + any text received after a command. Irrespective of whether + pipelining is used, the server MUST process commands in the order + they are sent. + + If the specific description of a command says it "MUST NOT be + pipelined", that command MUST end any pipeline of commands. That is, + the client MUST NOT send any following command until it receives the + CRLF at the end of the response from the command. The server MAY + ignore any data received after the command and before the CRLF at the + end of the response is sent to the client. + + The initial connection must not be part of a pipeline; that is, the + client MUST NOT send any command until it receives the CRLF at the + end of the greeting. + + If the client uses blocking system calls to send commands, it MUST + ensure that the amount of text sent in pipelining does not cause a + deadlock between transmission and reception. The amount of text + involved will depend on window sizes in the transmission layer; + typically, it is 4k octets for TCP. (Since the server only sends + data in response to commands from the client, the converse problem + does not occur.) + +3.5.1. Examples + + Example of correct use of pipelining: + + [C] GROUP misc.test + [C] STAT + [C] NEXT + [S] 211 1234 3000234 3002322 misc.test + [S] 223 3000234 <45223423@example.com> retrieved + [S] 223 3000237 <668929@example.org> retrieved + + Example of incorrect use of pipelining (the MODE READER command may + not be pipelined): + + [C] MODE READER + [C] DATE + [C] NEXT + [S] 200 Server ready, posting allowed + [S] 223 3000237 <668929@example.org> retrieved + + + +Feather Standards Track [Page 23] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The DATE command has been thrown away by the server, so there is no + 111 response to match it. + +3.6. Articles + + NNTP is intended to transfer articles between clients and servers. + For the purposes of this specification, articles are required to + conform to the rules in this section, and clients and servers MUST + correctly process any article received from the other that does so. + Note that this requirement applies only to the contents of + communications over NNTP; it does not prevent the client or server + from subsequently rejecting an article for reasons of local policy. + Also see Appendix A for further restrictions on the format of + articles in some uses of NNTP. + + An article consists of two parts: the headers and the body. They are + separated by a single empty line, or in other words by two + consecutive CRLF pairs (if there is more than one empty line, the + second and subsequent ones are part of the body). In order to meet + the general requirements of NNTP, an article MUST NOT include the + octet NUL, MUST NOT contain the octets LF and CR other than as part + of a CRLF pair, and MUST end with a CRLF pair. This specification + puts no further restrictions on the body; in particular, it MAY be + empty. + + The headers of an article consist of one or more header lines. Each + header line consists of a header name, a colon, a space, the header + content, and a CRLF, in that order. The name consists of one or more + printable US-ASCII characters other than colon and, for the purposes + of this specification, is not case sensitive. There MAY be more than + one header line with the same name. The content MUST NOT contain + CRLF; it MAY be empty. A header may be "folded"; that is, a CRLF + pair may be placed before any TAB or space in the line. There MUST + still be some other octet between any two CRLF pairs in a header + line. (Note that folding means that the header line occupies more + than one line when displayed or transmitted; nevertheless, it is + still referred to as "a" header line.) The presence or absence of + folding does not affect the meaning of the header line; that is, the + CRLF pairs introduced by folding are not considered part of the + header content. Header lines SHOULD NOT be folded before the space + after the colon that follows the header name and SHOULD include at + least one octet other than %x09 or %x20 between CRLF pairs. However, + if an article that fails to satisfy this requirement has been + received from elsewhere, clients and servers MAY transfer it to each + other without re-folding it. + + + + + + +Feather Standards Track [Page 24] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The content of a header SHOULD be in UTF-8. However, if an + implementation receives an article from elsewhere that uses octets in + the range 128 to 255 in some other manner, it MAY pass it to a client + or server without modification. Therefore, implementations MUST be + prepared to receive such headers, and data derived from them (e.g., + in the responses from the OVER command, Section 8.3), and MUST NOT + assume that they are always UTF-8. Any external processing of those + headers, including identifying the encoding used, is outside the + scope of this document. + + Each article MUST have a unique message-id; two articles offered by + an NNTP server MUST NOT have the same message-id. For the purposes + of this specification, message-ids are opaque strings that MUST meet + the following requirements: + + o A message-id MUST begin with "<", end with ">", and MUST NOT + contain the latter except at the end. + + o A message-id MUST be between 3 and 250 octets in length. + + o A message-id MUST NOT contain octets other than printable US-ASCII + characters. + + Two message-ids are the same if and only if they consist of the same + sequence of octets. + + This specification does not describe how the message-id of an article + is determined. If the server does not have any way to determine a + message-id from the article itself, it MUST synthesize one (this + specification does not require that the article be changed as a + result). See also Appendix A.2. + +4. The WILDMAT Format + + The WILDMAT format described here is based on the version first + developed by Rich Salz [SALZ1992], which was in turn derived from the + format used in the UNIX "find" command to articulate file names. It + was developed to provide a uniform mechanism for matching patterns in + the same manner that the UNIX shell matches filenames. + + + + + + + + + + + + +Feather Standards Track [Page 25] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +4.1. Wildmat Syntax + + A wildmat is described by the following ABNF [RFC4234] syntax, which + is an extract of that in Section 9.8. + + wildmat = wildmat-pattern *("," ["!"] wildmat-pattern) + wildmat-pattern = 1*wildmat-item + wildmat-item = wildmat-exact / wildmat-wild + wildmat-exact = %x22-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E / + UTF8-non-ascii ; exclude ! * , ? [ \ ] + wildmat-wild = "*" / "?" + + Note: the characters ",", "\", "[", and "]" are not allowed in + wildmats, while * and ? are always wildcards. This should not be a + problem, since these characters cannot occur in newsgroup names, + which is the only current use of wildmats. Backslash is commonly + used to suppress the special meaning of characters, whereas brackets + are used to introduce sets. However, these usages are not universal, + and interpretation of these characters in the context of UTF-8 + strings is potentially complex and differs from existing practice, so + they were omitted from this specification. A future extension to + this specification may provide semantics for these characters. + +4.2. Wildmat Semantics + + A wildmat is tested against a string and either matches or does not + match. To do this, each constituent is matched + against the string, and the rightmost pattern that matches is + identified. If that is not preceded with "!", the + whole wildmat matches. If it is preceded by "!", or if no matches, the whole wildmat does not match. + + For example, consider the wildmat "a*,!*b,*c*": + + o The string "aaa" matches because the rightmost match is with "a*". + + o The string "abb" does not match because the rightmost match is + with "*b". + + o The string "ccb" matches because the rightmost match is with + "*c*". + + o The string "xxx" does not match because no + matches. + + A matches a string if the string can be broken into + components, each of which matches the corresponding in + the pattern. The matches must be in the same order, and the whole + + + +Feather Standards Track [Page 26] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + string must be used in the match. The pattern is "anchored"; that + is, the first and last characters in the string must match the first + and last item, respectively (unless that item is an asterisk matching + zero characters). + + A matches the same character (which may be more than + one octet in UTF-8). + + "?" matches exactly one character (which may be more than one octet). + + "*" matches zero or more characters. It can match an empty string, + but it cannot match a subsequence of a UTF-8 sequence that is not + aligned to the character boundaries. + +4.3. Extensions + + An NNTP server or extension MAY extend the syntax or semantics of + wildmats provided that all wildmats that meet the requirements of + Section 4.1 have the meaning ascribed to them by Section 4.2. Future + editions of this document may also extend wildmats. + +4.4. Examples + + In these examples, $ and @ are used to represent the two octets %xC2 + and %xA3, respectively; $@ is thus the UTF-8 encoding for the pound + sterling symbol, shown as # in the descriptions. + + Wildmat Description of strings that match + abc The one string "abc" + abc,def The two strings "abc" and "def" + $@ The one character string "#" + a* Any string that begins with "a" + a*b Any string that begins with "a" and ends with "b" + a*,*b Any string that begins with "a" or ends with "b" + a*,!*b Any string that begins with "a" and does not end with + "b" + a*,!*b,c* Any string that begins with "a" and does not end with + "b", and any string that begins with "c" no matter + what it ends with + a*,c*,!*b Any string that begins with "a" or "c" and does not + end with "b" + ?a* Any string with "a" as its second character + ??a* Any string with "a" as its third character + *a? Any string with "a" as its penultimate character + *a?? Any string with "a" as its antepenultimate character + + + + + + +Feather Standards Track [Page 27] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +5. Session Administration Commands + +5.1. Initial Connection + +5.1.1. Usage + + This command MUST NOT be pipelined. + + Responses [1] + 200 Service available, posting allowed + 201 Service available, posting prohibited + 400 Service temporarily unavailable [2] + 502 Service permanently unavailable [2] + + [1] These are the only valid response codes for the initial greeting; + the server MUST not return any other generic response code. + + [2] Following a 400 or 502 response, the server MUST immediately + close the connection. + +5.1.2. Description + + There is no command presented by the client upon initial connection + to the server. The server MUST present an appropriate response code + as a greeting to the client. This response informs the client + whether service is available and whether the client is permitted to + post. + + If the server will accept further commands from the client including + POST, the server MUST present a 200 greeting code. If the server + will accept further commands from the client, but the client is not + authorized to post articles using the POST command, the server MUST + present a 201 greeting code. + + Otherwise, the server MUST present a 400 or 502 greeting code and + then immediately close the connection. 400 SHOULD be used if the + issue is only temporary (for example, because of load) and the client + can expect to be able to connect successfully at some point in the + future without making any changes. 502 MUST be used if the client is + not permitted under any circumstances to interact with the server, + and MAY be used if the server has insufficient information to + determine whether the issue is temporary or permanent. + + Note: the distinction between the 200 and 201 response codes has + turned out in practice to be insufficient; for example, some servers + do not allow posting until the client has authenticated, while other + clients assume that a 201 response means that posting will never be + possible even after authentication. Therefore, clients SHOULD use + + + +Feather Standards Track [Page 28] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + the CAPABILITIES command (Section 5.2) rather than rely on this + response. + +5.1.3. Examples + + Example of a normal connection from an authorized client that then + terminates the session (see Section 5.4): + + [Initial connection set-up completed.] + [S] 200 NNTP Service Ready, posting permitted + [C] QUIT + [S] 205 NNTP Service exits normally + [Server closes connection.] + + Example of a normal connection from an authorized client that is not + permitted to post, which also immediately terminates the session: + + [Initial connection set-up completed.] + [S] 201 NNTP Service Ready, posting prohibited + [C] QUIT + [S] 205 NNTP Service exits normally + [Server closes connection.] + + Example of a normal connection from an unauthorized client: + + [Initial connection set-up completed.] + [S] 502 NNTP Service permanently unavailable + [Server closes connection.] + + Example of a connection from a client if the server is unable to + provide service: + + [Initial connection set-up completed.] + [S] 400 NNTP Service temporarily unavailable + [Server closes connection.] + +5.2. CAPABILITIES + +5.2.1. Usage + + This command is mandatory. + + Syntax + CAPABILITIES [keyword] + + Responses + 101 Capability list follows (multi-line) + + + + +Feather Standards Track [Page 29] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Parameters + keyword additional feature, see description + +5.2.2. Description + + The CAPABILITIES command allows a client to determine the + capabilities of the server at any given time. + + This command MAY be issued at any time; the server MUST NOT require + it to be issued in order to make use of any capability. The response + generated by this command MAY change during a session because of + other state information (which, in turn, may be changed by the + effects of other commands or by external events). An NNTP client is + only able to get the current and correct information concerning + available capabilities at any point during a session by issuing a + CAPABILITIES command at that point of that session and processing the + response. + + The capability list is returned as a multi-line data block following + the 101 response code. Each capability is described by a separate + capability line. The server MUST NOT list the same capability twice + in the response, even with different arguments. Except that the + VERSION capability MUST be the first line, the order in which the + capability lines appears is not significant; the server need not even + consistently return the same order. + + While some capabilities are likely to be always available or never + available, others (notably extensions) will appear and disappear + depending on server state changes within the session or on external + events between sessions. An NNTP client MAY cache the results of + this command, but MUST NOT rely on the correctness of any cached + results, whether from earlier in this session or from a previous + session, MUST cope gracefully with the cached status being out of + date, and SHOULD (if caching results) provide a way to force the + cached information to be refreshed. Furthermore, a client MUST NOT + use cached results in relation to security, privacy, and + authentication extensions. See Section 12.6 for further discussion + of this topic. + + The keyword argument is not used by this specification. It is + provided so that extensions or revisions to this specification can + include extra features for this command without requiring the + CAPABILITIES command to be used twice (once to determine if the extra + features are available, and a second time to make use of them). If + the server does not recognise the argument (and it is a keyword), it + MUST respond with the 101 response code as if the argument had been + omitted. If an argument is provided that the server does recognise, + it MAY use the 101 response code or MAY use some other response code + + + +Feather Standards Track [Page 30] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + (which will be defined in the specification of that feature). If the + argument is not a keyword, the 501 generic response code MUST be + returned. The server MUST NOT generate any other response code to + the CAPABILITIES command. + +5.2.3. Examples + + Example of a minimal response (a read-only server): + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] LIST ACTIVE NEWSGROUPS + [S] . + + Example of a response from a server that has a range of facilities + and that also describes itself: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] IHAVE + [S] POST + [S] NEWNEWS + [S] LIST ACTIVE NEWSGROUPS ACTIVE.TIMES OVERVIEW.FMT + [S] IMPLEMENTATION INN 4.2 2004-12-25 + [S] OVER MSGID + [S] STREAMING + [S] XSECRET + [S] . + + Example of a server that supports more than one version of NNTP: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 3 + [S] READER + [S] LIST ACTIVE NEWSGROUPS + [S] . + + + + + + + + + + +Feather Standards Track [Page 31] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of a client attempting to use a feature of the CAPABILITIES + command that the server does not support: + + [C] CAPABILITIES AUTOUPDATE + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] IHAVE + [S] LIST ACTIVE NEWSGROUPS OVERVIEW.FMT HEADERS + [S] OVER MSGID + [S] HDR + [S] NEWNEWS + [S] . + +5.3. MODE READER + +5.3.1. Usage + + Indicating capability: MODE-READER + + This command MUST NOT be pipelined. + + Syntax + MODE READER + + Responses + 200 Posting allowed + 201 Posting prohibited + 502 Reading service permanently unavailable [1] + + [1] Following a 502 response the server MUST immediately close the + connection. + +5.3.2. Description + + The MODE READER command instructs a mode-switching server to switch + modes, as described in Section 3.4.2. + + If the server is mode-switching, it switches from its transit mode to + its reader mode, indicating this by changing the capability list + accordingly. It MUST then return a 200 or 201 response with the same + meaning as for the initial greeting (as described in Section 5.1.1). + Note that the response need not be the same as that presented during + the initial greeting. The client MUST NOT issue MODE READER more + than once in a session or after any security or privacy commands are + issued. When the MODE READER command is issued, the server MAY reset + its state to that immediately after the initial connection before + switching mode. + + + +Feather Standards Track [Page 32] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + If the server is not mode-switching, then the following apply: + + o If it advertises the READER capability, it MUST return a 200 or + 201 response with the same meaning as for the initial greeting; in + this case, the command MUST NOT affect the server state in any + way. + + o If it does not advertise the READER capability, it MUST return a + 502 response and then immediately close the connection. + +5.3.3. Examples + + Example of use of the MODE READER command on a transit-only server + (which therefore does not providing reading facilities): + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] IHAVE + [S] . + [C] MODE READER + [S] 502 Transit service only + [Server closes connection.] + + Example of use of the MODE READER command on a server that provides + reading facilities: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] LIST ACTIVE NEWSGROUPS + [S] . + [C] MODE READER + [S] 200 Reader mode, posting permitted + [C] IHAVE + [S] 500 Permission denied + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + + Note that in both of these situations, the client SHOULD NOT use MODE + READER. + + + + + + + + + +Feather Standards Track [Page 33] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of use of the MODE READER command on a mode-switching server: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] IHAVE + [S] MODE-READER + [S] . + [C] MODE READER + [S] 200 Reader mode, posting permitted + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] NEWNEWS + [S] LIST ACTIVE NEWSGROUPS + [S] STARTTLS + [S] . + + In this case, the server offers (but does not require) TLS privacy in + its reading mode but not in its transit mode. + + Example of use of the MODE READER command where the client is not + permitted to post: + + [C] MODE READER + [S] 201 NNTP Service Ready, posting prohibited + +5.4. QUIT + +5.4.1. Usage + + This command is mandatory. + + Syntax + QUIT + + Responses + 205 Connection closing + +5.4.2. Description + + The client uses the QUIT command to terminate the session. The + server MUST acknowledge the QUIT command and then close the + connection to the client. This is the preferred method for a client + to indicate that it has finished all of its transactions with the + NNTP server. + + + + +Feather Standards Track [Page 34] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + If a client simply disconnects (or if the connection times out or + some other fault occurs), the server MUST gracefully cease its + attempts to service the client, disconnecting from its end if + necessary. + + The server MUST NOT generate any response code to the QUIT command + other than 205 or, if any arguments are provided, 501. + +5.4.3. Examples + + [C] QUIT + [S] 205 closing connection + [Server closes connection.] + +6. Article Posting and Retrieval + + News-reading clients have available a variety of mechanisms to + retrieve articles via NNTP. The news articles are stored and indexed + using three types of keys. The first type of key is the message-id + of an article and is globally unique. The second type of key is + composed of a newsgroup name and an article number within that + newsgroup. On a particular server, there MUST only be one article + with a given number within any newsgroup, and an article MUST NOT + have two different numbers in the same newsgroup. An article can be + cross-posted to multiple newsgroups, so there may be multiple keys + that point to the same article on the same server; these MAY have + different numbers in each newsgroup. However, this type of key is + not required to be globally unique, so the same key MAY refer to + different articles on different servers. (Note that the terms + "group" and "newsgroup" are equivalent.) + + The final type of key is the arrival timestamp, giving the time that + the article arrived at the server. The server MUST ensure that + article numbers are issued in order of arrival timestamp; that is, + articles arriving later MUST have higher numbers than those that + arrive earlier. The server SHOULD allocate the next sequential + unused number to each new article. + + Article numbers MUST lie between 1 and 2,147,483,647, inclusive. The + client and server MAY use leading zeroes in specifying article + numbers but MUST NOT use more than 16 digits. In some situations, + the value zero replaces an article number to show some special + situation. + + Note that it is likely that the article number limit of 2,147,483,647 + will be increased by a future revision or extension to this + specification. While servers MUST NOT send article numbers greater + than this current limit, client and server developers are advised to + + + +Feather Standards Track [Page 35] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + use internal structures and datatypes capable of handling larger + values in anticipation of such a change. + +6.1. Group and Article Selection + + The following commands are used to set the "currently selected + newsgroup" and the "current article number", which are used by + various commands. At the start of an NNTP session, both of these + values are set to the special value "invalid". + +6.1.1. GROUP + +6.1.1.1. Usage + + Indicating capability: READER + + Syntax + GROUP group + + Responses + 211 number low high group Group successfully selected + 411 No such newsgroup + + Parameters + group Name of newsgroup + number Estimated number of articles in the group + low Reported low water mark + high Reported high water mark + +6.1.1.2. Description + + The GROUP command selects a newsgroup as the currently selected + newsgroup and returns summary information about it. + + The required argument is the name of the newsgroup to be selected + (e.g., "news.software.nntp"). A list of valid newsgroups may be + obtained by using the LIST ACTIVE command (see Section 7.6.3). + + The successful selection response will return the article numbers of + the first and last articles in the group at the moment of selection + (these numbers are referred to as the "reported low water mark" and + the "reported high water mark") and an estimate of the number of + articles in the group currently available. + + If the group is not empty, the estimate MUST be at least the actual + number of articles available and MUST be no greater than one more + than the difference between the reported low and high water marks. + (Some implementations will actually count the number of articles + + + +Feather Standards Track [Page 36] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + currently stored. Others will just subtract the low water mark from + the high water mark and add one to get an estimate.) + + If the group is empty, one of the following three situations will + occur. Clients MUST accept all three cases; servers MUST NOT + represent an empty group in any other way. + + o The high water mark will be one less than the low water mark, and + the estimated article count will be zero. Servers SHOULD use this + method to show an empty group. This is the only time that the + high water mark can be less than the low water mark. + + o All three numbers will be zero. + + o The high water mark is greater than or equal to the low water + mark. The estimated article count might be zero or non-zero; if + it is non-zero, the same requirements apply as for a non-empty + group. + + The set of articles in a group may change after the GROUP command is + carried out: + + o Articles may be removed from the group. + + o Articles may be reinstated in the group with the same article + number, but those articles MUST have numbers no less than the + reported low water mark (note that this is a reinstatement of the + previous article, not a new article reusing the number). + + o New articles may be added with article numbers greater than the + reported high water mark. (If an article that was the one with + the highest number has been removed and the high water mark has + been adjusted accordingly, the next new article will not have the + number one greater than the reported high water mark.) + + Except when the group is empty and all three numbers are zero, + whenever a subsequent GROUP command for the same newsgroup is issued, + either by the same client or a different client, the reported low + water mark in the response MUST be no less than that in any previous + response for that newsgroup in this session, and it SHOULD be no less + than that in any previous response for that newsgroup ever sent to + any client. Any failure to meet the latter condition SHOULD be + transient only. The client may make use of the low water mark to + remove all remembered information about articles with lower numbers, + as these will never recur. This includes the situation when the high + water mark is one less than the low water mark. No similar + assumption can be made about the high water mark, as this can + + + + +Feather Standards Track [Page 37] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + decrease if an article is removed and then increase again if it is + reinstated or if new articles arrive. + + When a valid group is selected by means of this command, the + currently selected newsgroup MUST be set to that group, and the + current article number MUST be set to the first article in the group + (this applies even if the group is already the currently selected + newsgroup). If an empty newsgroup is selected, the current article + number is made invalid. If an invalid group is specified, the + currently selected newsgroup and current article number MUST NOT be + changed. + + The GROUP or LISTGROUP command (see Section 6.1.2) MUST be used by a + client, and a successful response received, before any other command + is used that depends on the value of the currently selected newsgroup + or current article number. + + If the group specified is not available on the server, a 411 response + MUST be returned. + +6.1.1.3. Examples + + Example for a group known to the server: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + + Example for a group unknown to the server: + + [C] GROUP example.is.sob.bradner.or.barber + [S] 411 example.is.sob.bradner.or.barber is unknown + + Example of an empty group using the preferred response: + + [C] GROUP example.currently.empty.newsgroup + [S] 211 0 4000 3999 example.currently.empty.newsgroup + + Example of an empty group using an alternative response: + + [C] GROUP example.currently.empty.newsgroup + [S] 211 0 0 0 example.currently.empty.newsgroup + + Example of an empty group using a different alternative response: + + [C] GROUP example.currently.empty.newsgroup + [S] 211 0 4000 4321 example.currently.empty.newsgroup + + + + + +Feather Standards Track [Page 38] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example reselecting the currently selected newsgroup: + + [C] GROUP misc.test + [S] 211 1234 234 567 misc.test + [C] STAT 444 + [S] 223 444 <123456@example.net> retrieved + [C] GROUP misc.test + [S] 211 1234 234 567 misc.test + [C] STAT + [S] 223 234 retrieved + +6.1.2. LISTGROUP + +6.1.2.1. Usage + + Indicating capability: READER + + Syntax + LISTGROUP [group [range]] + + Responses + 211 number low high group Article numbers follow (multi-line) + 411 No such newsgroup + 412 No newsgroup selected [1] + + Parameters + group Name of newsgroup + range Range of articles to report + number Estimated number of articles in the group + low Reported low water mark + high Reported high water mark + + [1] The 412 response can only occur if no group has been specified. + +6.1.2.2. Description + + The LISTGROUP command selects a newsgroup in the same manner as the + GROUP command (see Section 6.1.1) but also provides a list of article + numbers in the newsgroup. If no group is specified, the currently + selected newsgroup is used. + + On success, a list of article numbers is returned as a multi-line + data block following the 211 response code (the arguments on the + initial response line are the same as for the GROUP command). The + list contains one number per line and is in numerical order. It + lists precisely those articles that exist in the group at the moment + of selection (therefore, an empty group produces an empty list). If + the optional range argument is specified, only articles within the + + + +Feather Standards Track [Page 39] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + range are included in the list (therefore, the list MAY be empty even + if the group is not). + + The range argument may be any of the following: + + o An article number. + + o An article number followed by a dash to indicate all following. + + o An article number followed by a dash followed by another article + number. + + In the last case, if the second number is less than the first number, + then the range contains no articles. Omitting the range is + equivalent to the range 1- being specified. + + If the group specified is not available on the server, a 411 response + MUST be returned. If no group is specified and the currently + selected newsgroup is invalid, a 412 response MUST be returned. + + Except that the group argument is optional, that a range argument can + be specified, and that a multi-line data block follows the 211 + response code, the LISTGROUP command is identical to the GROUP + command. In particular, when successful, the command sets the + current article number to the first article in the group, if any, + even if this is not within the range specified by the second + argument. + + Note that the range argument is a new feature in this specification + and servers that do not support CAPABILITIES (and therefore do not + conform to this specification) are unlikely to support it. + +6.1.2.3. Examples + + Example of LISTGROUP being used to select a group: + + [C] LISTGROUP misc.test + [S] 211 2000 3000234 3002322 misc.test list follows + [S] 3000234 + [S] 3000237 + [S] 3000238 + [S] 3000239 + [S] 3002322 + [S] . + + + + + + + +Feather Standards Track [Page 40] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of LISTGROUP on an empty group: + + [C] LISTGROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup list follows + [S] . + + Example of LISTGROUP on a valid, currently selected newsgroup: + + [C] GROUP misc.test + [S] 211 2000 3000234 3002322 misc.test + [C] LISTGROUP + [S] 211 2000 3000234 3002322 misc.test list follows + [S] 3000234 + [S] 3000237 + [S] 3000238 + [S] 3000239 + [S] 3002322 + [S] . + + Example of LISTGROUP with a range: + + [C] LISTGROUP misc.test 3000238-3000248 + [S] 211 2000 3000234 3002322 misc.test list follows + [S] 3000238 + [S] 3000239 + [S] . + + Example of LISTGROUP with an empty range: + + [C] LISTGROUP misc.test 12345678- + [S] 211 2000 3000234 3002322 misc.test list follows + [S] . + + Example of LISTGROUP with an invalid range: + + [C] LISTGROUP misc.test 9999-111 + [S] 211 2000 3000234 3002322 misc.test list follows + [S] . + + + + + + + + + + + + + +Feather Standards Track [Page 41] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.1.3. LAST + +6.1.3.1. Usage + + Indicating capability: READER + + Syntax + LAST + + Responses + 223 n message-id Article found + 412 No newsgroup selected + 420 Current article number is invalid + 422 No previous article in this group + + Parameters + n Article number + message-id Article message-id + +6.1.3.2. Description + + If the currently selected newsgroup is valid, the current article + number MUST be set to the previous article in that newsgroup (that + is, the highest existing article number less than the current article + number). If successful, a response indicating the new current + article number and the message-id of that article MUST be returned. + No article text is sent in response to this command. + + There MAY be no previous article in the group, although the current + article number is not the reported low water mark. There MUST NOT be + a previous article when the current article number is the reported + low water mark. + + Because articles can be removed and added, the results of multiple + LAST and NEXT commands MAY not be consistent over the life of a + particular NNTP session. + + If the current article number is already the first article of the + newsgroup, a 422 response MUST be returned. If the current article + number is invalid, a 420 response MUST be returned. If the currently + selected newsgroup is invalid, a 412 response MUST be returned. In + all three cases, the currently selected newsgroup and current article + number MUST NOT be altered. + + + + + + + + +Feather Standards Track [Page 42] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.1.3.3. Examples + + Example of a successful article retrieval using LAST: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] NEXT + [S] 223 3000237 <668929@example.org> retrieved + [C] LAST + [S] 223 3000234 <45223423@example.com> retrieved + + Example of an attempt to retrieve an article without having selected + a group (via the GROUP command) first: + + [Assumes currently selected newsgroup is invalid.] + [C] LAST + [S] 412 no newsgroup selected + + Example of an attempt to retrieve an article using the LAST command + when the current article number is that of the first article in the + group: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] LAST + [S] 422 No previous article to retrieve + + Example of an attempt to retrieve an article using the LAST command + when the currently selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] LAST + [S] 420 No current article selected + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 43] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.1.4. NEXT + +6.1.4.1. Usage + + Indicating capability: READER + + Syntax + NEXT + + Responses + 223 n message-id Article found + 412 No newsgroup selected + 420 Current article number is invalid + 421 No next article in this group + + Parameters + n Article number + message-id Article message-id + +6.1.4.2. Description + + If the currently selected newsgroup is valid, the current article + number MUST be set to the next article in that newsgroup (that is, + the lowest existing article number greater than the current article + number). If successful, a response indicating the new current + article number and the message-id of that article MUST be returned. + No article text is sent in response to this command. + + If the current article number is already the last article of the + newsgroup, a 421 response MUST be returned. In all other aspects + (apart, of course, from the lack of 422 response), this command is + identical to the LAST command (Section 6.1.3). + +6.1.4.3. Examples + + Example of a successful article retrieval using NEXT: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] NEXT + [S] 223 3000237 <668929@example.org> retrieved + + + + + + + + + + +Feather Standards Track [Page 44] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an attempt to retrieve an article without having selected + a group (via the GROUP command) first: + + [Assumes currently selected newsgroup is invalid.] + [C] NEXT + [S] 412 no newsgroup selected + + Example of an attempt to retrieve an article using the NEXT command + when the current article number is that of the last article in the + group: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] STAT 3002322 + [S] 223 3002322 <411@example.net> retrieved + [C] NEXT + [S] 421 No next article to retrieve + + Example of an attempt to retrieve an article using the NEXT command + when the currently selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] NEXT + [S] 420 No current article selected + +6.2. Retrieval of Articles and Article Sections + + The ARTICLE, BODY, HEAD, and STAT commands are very similar. They + differ only in the parts of the article that are presented to the + client and in the successful response code. The ARTICLE command is + described here in full, while the other three commands are described + in terms of the differences. As specified in Section 3.6, an article + consists of two parts: the article headers and the article body. + + When responding to one of these commands, the server MUST present the + entire article or appropriate part and MUST NOT attempt to alter or + translate it in any way. + + + + + + + + + + + + + +Feather Standards Track [Page 45] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.2.1. ARTICLE + +6.2.1.1. Usage + + Indicating capability: READER + + Syntax + ARTICLE message-id + ARTICLE number + ARTICLE + + Responses + + First form (message-id specified) + 220 0|n message-id Article follows (multi-line) + 430 No article with that message-id + + Second form (article number specified) + 220 n message-id Article follows (multi-line) + 412 No newsgroup selected + 423 No article with that number + + Third form (current article number used) + 220 n message-id Article follows (multi-line) + 412 No newsgroup selected + 420 Current article number is invalid + + Parameters + number Requested article number + n Returned article number + message-id Article message-id + +6.2.1.2. Description + + The ARTICLE command selects an article according to the arguments and + presents the entire article (that is, the headers, an empty line, and + the body, in that order) to the client. The command has three forms. + + In the first form, a message-id is specified, and the server presents + the article with that message-id. In this case, the server MUST NOT + alter the currently selected newsgroup or current article number. + This is both to facilitate the presentation of articles that may be + referenced within another article being read, and because of the + semantic difficulties of determining the proper sequence and + membership of an article that may have been cross-posted to more than + one newsgroup. + + + + + +Feather Standards Track [Page 46] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + In the response, the article number MUST be replaced with zero, + unless there is a currently selected newsgroup and the article is + present in that group, in which case the server MAY use the article's + number in that group. (The server is not required to determine + whether the article is in the currently selected newsgroup or, if so, + what article number it has; the client MUST always be prepared for + zero to be specified.) The server MUST NOT provide an article number + unless use of that number in a second ARTICLE command immediately + following this one would return the same article. Even if the server + chooses to return article numbers in these circumstances, it need not + do so consistently; it MAY return zero to any such command (also see + the STAT examples, Section 6.2.4.3). + + In the second form, an article number is specified. If there is an + article with that number in the currently selected newsgroup, the + server MUST set the current article number to that number. + + In the third form, the article indicated by the current article + number in the currently selected newsgroup is used. + + Note that a previously valid article number MAY become invalid if the + article has been removed. A previously invalid article number MAY + become valid if the article has been reinstated, but this article + number MUST be no less than the reported low water mark for that + group. + + The server MUST NOT change the currently selected newsgroup as a + result of this command. The server MUST NOT change the current + article number except when an article number argument was provided + and the article exists; in particular, it MUST NOT change it + following an unsuccessful response. + + Since the message-id is unique for each article, it may be used by a + client to skip duplicate displays of articles that have been posted + more than once, or to more than one newsgroup. + + The article is returned as a multi-line data block following the 220 + response code. + + If the argument is a message-id and no such article exists, a 430 + response MUST be returned. If the argument is a number or is omitted + and the currently selected newsgroup is invalid, a 412 response MUST + be returned. If the argument is a number and that article does not + exist in the currently selected newsgroup, a 423 response MUST be + returned. If the argument is omitted and the current article number + is invalid, a 420 response MUST be returned. + + + + + +Feather Standards Track [Page 47] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.2.1.3. Examples + + Example of a successful retrieval of an article (explicitly not using + an article number): + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] ARTICLE + [S] 220 3000234 <45223423@example.com> + [S] Path: pathost!demo!whitehouse!not-for-mail + [S] From: "Demo User" + [S] Newsgroups: misc.test + [S] Subject: I am just a test article + [S] Date: 6 Oct 1998 04:38:40 -0500 + [S] Organization: An Example Net, Uncertain, Texas + [S] Message-ID: <45223423@example.com> + [S] + [S] This is just a test article. + [S] . + + Example of a successful retrieval of an article by message-id: + + [C] ARTICLE <45223423@example.com> + [S] 220 0 <45223423@example.com> + [S] Path: pathost!demo!whitehouse!not-for-mail + [S] From: "Demo User" + [S] Newsgroups: misc.test + [S] Subject: I am just a test article + [S] Date: 6 Oct 1998 04:38:40 -0500 + [S] Organization: An Example Net, Uncertain, Texas + [S] Message-ID: <45223423@example.com> + [S] + [S] This is just a test article. + [S] . + + Example of an unsuccessful retrieval of an article by message-id: + + [C] ARTICLE + [S] 430 No Such Article Found + + Example of an unsuccessful retrieval of an article by number: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 news.groups + [C] ARTICLE 300256 + [S] 423 No article with that number + + + + + +Feather Standards Track [Page 48] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an unsuccessful retrieval of an article by number because + no newsgroup was selected first: + + [Assumes currently selected newsgroup is invalid.] + [C] ARTICLE 300256 + [S] 412 No newsgroup selected + + Example of an attempt to retrieve an article when the currently + selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] ARTICLE + [S] 420 No current article selected + +6.2.2. HEAD + +6.2.2.1. Usage + + This command is mandatory. + + Syntax + HEAD message-id + HEAD number + HEAD + + Responses + + First form (message-id specified) + 221 0|n message-id Headers follow (multi-line) + 430 No article with that message-id + + Second form (article number specified) + 221 n message-id Headers follow (multi-line) + 412 No newsgroup selected + 423 No article with that number + + Third form (current article number used) + 221 n message-id Headers follow (multi-line) + 412 No newsgroup selected + 420 Current article number is invalid + + Parameters + number Requested article number + n Returned article number + message-id Article message-id + + + + + +Feather Standards Track [Page 49] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.2.2.2. Description + + The HEAD command behaves identically to the ARTICLE command except + that, if the article exists, the response code is 221 instead of 220 + and only the headers are presented (the empty line separating the + headers and body MUST NOT be included). + +6.2.2.3. Examples + + Example of a successful retrieval of the headers of an article + (explicitly not using an article number): + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HEAD + [S] 221 3000234 <45223423@example.com> + [S] Path: pathost!demo!whitehouse!not-for-mail + [S] From: "Demo User" + [S] Newsgroups: misc.test + [S] Subject: I am just a test article + [S] Date: 6 Oct 1998 04:38:40 -0500 + [S] Organization: An Example Net, Uncertain, Texas + [S] Message-ID: <45223423@example.com> + [S] . + + Example of a successful retrieval of the headers of an article by + message-id: + + [C] HEAD <45223423@example.com> + [S] 221 0 <45223423@example.com> + [S] Path: pathost!demo!whitehouse!not-for-mail + [S] From: "Demo User" + [S] Newsgroups: misc.test + [S] Subject: I am just a test article + [S] Date: 6 Oct 1998 04:38:40 -0500 + [S] Organization: An Example Net, Uncertain, Texas + [S] Message-ID: <45223423@example.com> + [S] . + + Example of an unsuccessful retrieval of the headers of an article by + message-id: + + [C] HEAD + [S] 430 No Such Article Found + + + + + + + +Feather Standards Track [Page 50] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an unsuccessful retrieval of the headers of an article by + number: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HEAD 300256 + [S] 423 No article with that number + + Example of an unsuccessful retrieval of the headers of an article by + number because no newsgroup was selected first: + + [Assumes currently selected newsgroup is invalid.] + [C] HEAD 300256 + [S] 412 No newsgroup selected + + Example of an attempt to retrieve the headers of an article when the + currently selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] HEAD + [S] 420 No current article selected + +6.2.3. BODY + +6.2.3.1. Usage + + Indicating capability: READER + + Syntax + BODY message-id + BODY number + BODY + + Responses + + First form (message-id specified) + 222 0|n message-id Body follows (multi-line) + 430 No article with that message-id + + Second form (article number specified) + 222 n message-id Body follows (multi-line) + 412 No newsgroup selected + 423 No article with that number + + + + + + + +Feather Standards Track [Page 51] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Third form (current article number used) + 222 n message-id Body follows (multi-line) + 412 No newsgroup selected + 420 Current article number is invalid + + Parameters + number Requested article number + n Returned article number + message-id Article message-id + +6.2.3.2. Description + + The BODY command behaves identically to the ARTICLE command except + that, if the article exists, the response code is 222 instead of 220 + and only the body is presented (the empty line separating the headers + and body MUST NOT be included). + +6.2.3.3. Examples + + Example of a successful retrieval of the body of an article + (explicitly not using an article number): + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] BODY + [S] 222 3000234 <45223423@example.com> + [S] This is just a test article. + [S] . + + Example of a successful retrieval of the body of an article by + message-id: + + [C] BODY <45223423@example.com> + [S] 222 0 <45223423@example.com> + [S] This is just a test article. + [S] . + + Example of an unsuccessful retrieval of the body of an article by + message-id: + + [C] BODY + [S] 430 No Such Article Found + + + + + + + + + +Feather Standards Track [Page 52] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an unsuccessful retrieval of the body of an article by + number: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] BODY 300256 + [S] 423 No article with that number + + Example of an unsuccessful retrieval of the body of an article by + number because no newsgroup was selected first: + + [Assumes currently selected newsgroup is invalid.] + [C] BODY 300256 + [S] 412 No newsgroup selected + + Example of an attempt to retrieve the body of an article when the + currently selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] BODY + [S] 420 No current article selected + +6.2.4. STAT + +6.2.4.1. Usage + + This command is mandatory. + + Syntax + STAT message-id + STAT number + STAT + + Responses + + First form (message-id specified) + 223 0|n message-id Article exists + 430 No article with that message-id + + Second form (article number specified) + 223 n message-id Article exists + 412 No newsgroup selected + 423 No article with that number + + + + + + + +Feather Standards Track [Page 53] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Third form (current article number used) + 223 n message-id Article exists + 412 No newsgroup selected + 420 Current article number is invalid + + Parameters + number Requested article number + n Returned article number + message-id Article message-id + +6.2.4.2. Description + + The STAT command behaves identically to the ARTICLE command except + that, if the article exists, it is NOT presented to the client and + the response code is 223 instead of 220. Note that the response is + NOT multi-line. + + This command allows the client to determine whether an article exists + and, in the second and third forms, what its message-id is, without + having to process an arbitrary amount of text. + +6.2.4.3. Examples + + Example of STAT on an existing article (explicitly not using an + article number): + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] STAT + [S] 223 3000234 <45223423@example.com> + + Example of STAT on an existing article by message-id: + + [C] STAT <45223423@example.com> + [S] 223 0 <45223423@example.com> + + Example of STAT on an article not on the server by message-id: + + [C] STAT + [S] 430 No Such Article Found + + Example of STAT on an article not in the server by number: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] STAT 300256 + [S] 423 No article with that number + + + + +Feather Standards Track [Page 54] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of STAT on an article by number when no newsgroup was + selected first: + + [Assumes currently selected newsgroup is invalid.] + [C] STAT 300256 + [S] 412 No newsgroup selected + + Example of STAT on an article when the currently selected newsgroup + is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] STAT + [S] 420 No current article selected + + Example of STAT by message-id on a server that sometimes reports the + actual article number: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] STAT + [S] 223 3000234 <45223423@example.com> + [C] STAT <45223423@example.com> + [S] 223 0 <45223423@example.com> + [C] STAT <45223423@example.com> + [S] 223 3000234 <45223423@example.com> + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] STAT <45223423@example.com> + [S] 223 0 <45223423@example.com> + [C] GROUP alt.crossposts + [S] 211 9999 111111 222222 alt.crossposts + [C] STAT <45223423@example.com> + [S] 223 123456 <45223423@example.com> + [C] STAT + [S] 223 111111 <23894720@example.com> + + The first STAT command establishes the identity of an article in the + group. The second and third show that the server may, but need not, + give the article number when the message-id is specified. The fourth + STAT command shows that zero must be specified if the article isn't + in the currently selected newsgroup. The fifth shows that the + number, if provided, must be that relating to the currently selected + newsgroup. The last one shows that the current article number is + still not changed by the use of STAT with a message-id even if it + returns an article number. + + + + + +Feather Standards Track [Page 55] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.3. Article Posting + + Article posting is done in one of two ways: individual article + posting from news-reading clients using POST, and article transfer + from other news servers using IHAVE. + +6.3.1. POST + +6.3.1.1. Usage + + Indicating capability: POST + + This command MUST NOT be pipelined. + + Syntax + POST + + Responses + + Initial responses + 340 Send article to be posted + 440 Posting not permitted + + Subsequent responses + 240 Article received OK + 441 Posting failed + +6.3.1.2. Description + + If posting is allowed, a 340 response MUST be returned to indicate + that the article to be posted should be sent. If posting is + prohibited for some installation-dependent reason, a 440 response + MUST be returned. + + If posting is permitted, the article MUST be in the format specified + in Section 3.6 and MUST be sent by the client to the server as a + multi-line data block (see Section 3.1.1). Thus a single dot (".") + on a line indicates the end of the text, and lines starting with a + dot in the original text have that dot doubled during transmission. + + Following the presentation of the termination sequence by the client, + the server MUST return a response indicating success or failure of + the article transfer. Note that response codes 340 and 440 are used + in direct response to the POST command while 240 and 441 are returned + after the article is sent. + + + + + + +Feather Standards Track [Page 56] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + A response of 240 SHOULD indicate that, barring unforeseen server + errors, the posted article will be made available on the server + and/or transferred to other servers, as appropriate, possibly + following further processing. In other words, articles not wanted by + the server SHOULD be rejected with a 441 response, rather than being + accepted and then discarded silently. However, the client SHOULD NOT + assume that the article has been successfully transferred unless it + receives an affirmative response from the server and SHOULD NOT + assume that it is being made available to other clients without + explicitly checking (for example, using the STAT command). + + If the session is interrupted before the response is received, it is + possible that an affirmative response was sent but has been lost. + Therefore, in any subsequent session, the client SHOULD either check + whether the article was successfully posted before resending or + ensure that the server will allocate the same message-id to the new + attempt (see Appendix A.2). The latter approach is preferred since + the article might not have been made available for reading yet (for + example, it may have to go through a moderation process). + +6.3.1.3. Examples + + Example of a successful posting: + + [C] POST + [S] 340 Input article; end with . + [C] From: "Demo User" + [C] Newsgroups: misc.test + [C] Subject: I am just a test article + [C] Organization: An Example Net + [C] + [C] This is just a test article. + [C] . + [S] 240 Article received OK + + Example of an unsuccessful posting: + + [C] POST + [S] 340 Input article; end with . + [C] From: "Demo User" + [C] Newsgroups: misc.test + [C] Subject: I am just a test article + [C] Organization: An Example Net + [C] + [C] This is just a test article. + [C] . + [S] 441 Posting failed + + + + +Feather Standards Track [Page 57] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an attempt to post when posting is not allowed: + + [Initial connection set-up completed.] + [S] 201 NNTP Service Ready, posting prohibited + [C] POST + [S] 440 Posting not permitted + +6.3.2. IHAVE + +6.3.2.1. Usage + + Indicating capability: IHAVE + + This command MUST NOT be pipelined. + + Syntax + IHAVE message-id + + Responses + + Initial responses + 335 Send article to be transferred + 435 Article not wanted + 436 Transfer not possible; try again later + + Subsequent responses + 235 Article transferred OK + 436 Transfer failed; try again later + 437 Transfer rejected; do not retry + + Parameters + message-id Article message-id + +6.3.2.2. Description + + The IHAVE command informs the server that the client has an article + with the specified message-id. If the server desires a copy of that + article, a 335 response MUST be returned, instructing the client to + send the entire article. If the server does not want the article + (if, for example, the server already has a copy of it), a 435 + response MUST be returned, indicating that the article is not wanted. + Finally, if the article isn't wanted immediately but the client + should retry later if possible (if, for example, another client is in + the process of sending the same article to the server), a 436 + response MUST be returned. + + + + + + +Feather Standards Track [Page 58] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + If transmission of the article is requested, the client MUST send the + entire article, including headers and body, to the server as a + multi-line data block (see Section 3.1.1). Thus, a single dot (".") + on a line indicates the end of the text, and lines starting with a + dot in the original text have that dot doubled during transmission. + The server MUST return a 235 response, indicating that the article + was successfully transferred; a 436 response, indicating that the + transfer failed but should be tried again later; or a 437 response, + indicating that the article was rejected. + + This function differs from the POST command in that it is intended + for use in transferring already-posted articles between hosts. It + SHOULD NOT be used when the client is a personal news-reading + program, since use of this command indicates that the article has + already been posted at another site and is simply being forwarded + from another host. However, despite this, the server MAY elect not + to post or forward the article if, after further examination of the + article, it deems it inappropriate to do so. Reasons for such + subsequent rejection of an article may include problems such as + inappropriate newsgroups or distributions, disc space limitations, + article lengths, garbled headers, and the like. These are typically + restrictions enforced by the server host's news software and not + necessarily by the NNTP server itself. + + The client SHOULD NOT assume that the article has been successfully + transferred unless it receives an affirmative response from the + server. A lack of response (such as a dropped network connection or + a network timeout) SHOULD be treated the same as a 436 response. + + Because some news server software may not immediately be able to + determine whether an article is suitable for posting or forwarding, + an NNTP server MAY acknowledge the successful transfer of the article + (with a 235 response) but later silently discard it. + + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 59] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +6.3.2.3. Examples + + Example of successfully sending an article to another site: + + [C] IHAVE + [S] 335 Send it; end with . + [C] Path: pathost!demo!somewhere!not-for-mail + [C] From: "Demo User" + [C] Newsgroups: misc.test + [C] Subject: I am just a test article + [C] Date: 6 Oct 1998 04:38:40 -0500 + [C] Organization: An Example Com, San Jose, CA + [C] Message-ID: + [C] + [C] This is just a test article. + [C] . + [S] 235 Article transferred OK + + Example of sending an article to another site that rejects it. Note + that the message-id in the IHAVE command is not the same as the one + in the article headers; while this is bad practice and SHOULD NOT be + done, it is not forbidden. + + [C] IHAVE + [S] 335 Send it; end with . + [C] Path: pathost!demo!somewhere!not-for-mail + [C] From: "Demo User" + [C] Newsgroups: misc.test + [C] Subject: I am just a test article + [C] Date: 6 Oct 1998 04:38:40 -0500 + [C] Organization: An Example Com, San Jose, CA + [C] Message-ID: + [C] + [C] This is just a test article. + [C] . + [S] 437 Article rejected; don't send again + + + + + + + + + + + + + + + +Feather Standards Track [Page 60] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of sending an article to another site where the transfer + fails: + + [C] IHAVE + [S] 335 Send it; end with . + [C] Path: pathost!demo!somewhere!not-for-mail + [C] From: "Demo User" + [C] Newsgroups: misc.test + [C] Subject: I am just a test article + [C] Date: 6 Oct 1998 04:38:40 -0500 + [C] Organization: An Example Com, San Jose, CA + [C] Message-ID: + [C] + [C] This is just a test article. + [C] . + [S] 436 Transfer failed + + Example of sending an article to a site that already has it: + + [C] IHAVE + [S] 435 Duplicate + + Example of sending an article to a site that requests that the + article be tried again later: + + [C] IHAVE + [S] 436 Retry later + +7. Information Commands + + This section lists other commands that may be used at any time + between the beginning of a session and its termination. Using these + commands does not alter any state information, but the response + generated from their use may provide useful information to clients. + +7.1. DATE + +7.1.1. Usage + + Indicating capability: READER + + Syntax + DATE + + Responses + 111 yyyymmddhhmmss Server date and time + + + + + +Feather Standards Track [Page 61] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Parameters + yyyymmddhhmmss Current UTC date and time on server + +7.1.2. Description + + This command exists to help clients find out the current Coordinated + Universal Time [TF.686-1] from the server's perspective. This + command SHOULD NOT be used as a substitute for NTP [RFC1305] but to + provide information that might be useful when using the NEWNEWS + command (see Section 7.4). + + The DATE command MUST return a timestamp from the same clock as is + used for determining article arrival and group creation times (see + Section 6). This clock SHOULD be monotonic, and adjustments SHOULD + be made by running it fast or slow compared to "real" time rather + than by making sudden jumps. A system providing NNTP service SHOULD + keep the system clock as accurate as possible, either with NTP or by + some other method. + + The server MUST return a 111 response specifying the date and time on + the server in the form yyyymmddhhmmss. This date and time is in + Coordinated Universal Time. + +7.1.3. Examples + + [C] DATE + [S] 111 19990623135624 + +7.2. HELP + +7.2.1. Usage + + This command is mandatory. + + Syntax + HELP + + Responses + 100 Help text follows (multi-line) + +7.2.2. Description + + This command provides a short summary of the commands that are + understood by this implementation of the server. The help text will + be presented as a multi-line data block following the 100 response + code. + + + + + +Feather Standards Track [Page 62] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + This text is not guaranteed to be in any particular format (but must + be UTF-8) and MUST NOT be used by clients as a replacement for the + CAPABILITIES command described in Section 5.2. + +7.2.3. Examples + + [C] HELP + [S] 100 Help text follows + [S] This is some help text. There is no specific + [S] formatting requirement for this test, though + [S] it is customary for it to list the valid commands + [S] and give a brief definition of what they do. + [S] . + +7.3. NEWGROUPS + +7.3.1. Usage + + Indicating capability: READER + + Syntax + NEWGROUPS date time [GMT] + + Responses + 231 List of new newsgroups follows (multi-line) + + Parameters + date Date in yymmdd or yyyymmdd format + time Time in hhmmss format + +7.3.2. Description + + This command returns a list of newsgroups created on the server since + the specified date and time. The results are in the same format as + the LIST ACTIVE command (see Section 7.6.3). However, they MAY + include groups not available on the server (and so not returned by + LIST ACTIVE) and MAY omit groups for which the creation date is not + available. + + The date is specified as 6 or 8 digits in the format [xx]yymmdd, + where xx is the first two digits of the year (19-99), yy is the last + two digits of the year (00-99), mm is the month (01-12), and dd is + the day of the month (01-31). Clients SHOULD specify all four digits + of the year. If the first two digits of the year are not specified + (this is supported only for backward compatibility), the year is to + be taken from the current century if yy is smaller than or equal to + the current year, and the previous century otherwise. + + + + +Feather Standards Track [Page 63] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The time is specified as 6 digits in the format hhmmss, where hh is + the hours in the 24-hour clock (00-23), mm is the minutes (00-59), + and ss is the seconds (00-60, to allow for leap seconds). The token + "GMT" specifies that the date and time are given in Coordinated + Universal Time [TF.686-1]; if it is omitted, then the date and time + are specified in the server's local timezone. Note that there is no + way of using the protocol specified in this document to establish the + server's local timezone. + + Note that an empty list is a possible valid response and indicates + that there are no new newsgroups since that date-time. + + Clients SHOULD make all queries using Coordinated Universal Time + (i.e., by including the "GMT" argument) when possible. + +7.3.3. Examples + + Example where there are new groups: + + [C] NEWGROUPS 19990624 000000 GMT + [S] 231 list of new newsgroups follows + [S] alt.rfc-writers.recovery 4 1 y + [S] tx.natives.recovery 89 56 y + [S] . + + Example where there are no new groups: + + [C] NEWGROUPS 19990624 000000 GMT + [S] 231 list of new newsgroups follows + [S] . + +7.4. NEWNEWS + +7.4.1. Usage + + Indicating capability: NEWNEWS + + Syntax + NEWNEWS wildmat date time [GMT] + + Responses + 230 List of new articles follows (multi-line) + + Parameters + wildmat Newsgroups of interest + date Date in yymmdd or yyyymmdd format + time Time in hhmmss format + + + + +Feather Standards Track [Page 64] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +7.4.2. Description + + This command returns a list of message-ids of articles posted or + received on the server, in the newsgroups whose names match the + wildmat, since the specified date and time. One message-id is sent + on each line; the order of the response has no specific significance + and may vary from response to response in the same session. A + message-id MAY appear more than once; if it does, it has the same + meaning as if it appeared only once. + + Date and time are in the same format as the NEWGROUPS command (see + Section 7.3). + + Note that an empty list is a possible valid response and indicates + that there is currently no new news in the relevant groups. + + Clients SHOULD make all queries in Coordinated Universal Time (i.e., + by using the "GMT" argument) when possible. + +7.4.3. Examples + + Example where there are new articles: + + [C] NEWNEWS news.*,sci.* 19990624 000000 GMT + [S] 230 list of new articles by message-id follows + [S] + [S] + [S] . + + Example where there are no new articles: + + [C] NEWNEWS alt.* 19990624 000000 GMT + [S] 230 list of new articles by message-id follows + [S] . + +7.5. Time + + As described in Section 6, each article has an arrival timestamp. + Each newsgroup also has a creation timestamp. These timestamps are + used by the NEWNEWS and NEWGROUP commands to construct their + responses. + + Clients can ensure that they do not have gaps in lists of articles or + groups by using the DATE command in the following manner: + + First session: + Issue DATE command and record result. + Issue NEWNEWS command using a previously chosen timestamp. + + + +Feather Standards Track [Page 65] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Subsequent sessions: + Issue DATE command and hold result in temporary storage. + Issue NEWNEWS command using timestamp saved from previous session. + Overwrite saved timestamp with that currently in temporary + storage. + + In order to allow for minor errors, clients MAY want to adjust the + timestamp back by two or three minutes before using it in NEWNEWS. + +7.5.1. Examples + + First session: + + [C] DATE + [S] 111 20010203112233 + [C] NEWNEWS local.chat 20001231 235959 GMT + [S] 230 list follows + [S] + [S] + [S] + [S] . + + Second session (the client has subtracted 3 minutes from the + timestamp returned previously): + + [C] DATE + [S] 111 20010204003344 + [C] NEWNEWS local.chat 20010203 111933 GMT + [S] 230 list follows + [S] + [S] + [S] + [S] . + + Note how arrived in the 3 minute gap and so + is listed in both responses. + +7.6. The LIST Commands + + The LIST family of commands all return information that is multi-line + and that can, in general, be expected not to change during the + session. Often the information is related to newsgroups, in which + case the response has one line per newsgroup and a wildmat MAY be + provided to restrict the groups for which information is returned. + + The set of available keywords (including those provided by + extensions) is given in the capability list with capability label + LIST. + + + +Feather Standards Track [Page 66] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +7.6.1. LIST + +7.6.1.1. Usage + + Indicating capability: LIST + + Syntax + LIST [keyword [wildmat|argument]] + + Responses + 215 Information follows (multi-line) + + Parameters + keyword Information requested [1] + argument Specific to keyword + wildmat Groups of interest + + [1] If no keyword is provided, it defaults to ACTIVE. + +7.6.1.2. Description + + The LIST command allows the server to provide blocks of information + to the client. This information may be global or may be related to + newsgroups; in the latter case, the information may be returned + either for all groups or only for those matching a wildmat. Each + block of information is represented by a different keyword. The + command returns the specific information identified by the keyword. + + If the information is available, it is returned as a multi-line data + block following the 215 response code. The format of the information + depends on the keyword. The information MAY be affected by the + additional argument, but the format MUST NOT be. + + If the information is based on newsgroups and the optional wildmat + argument is specified, the response is limited to only the groups (if + any) whose names match the wildmat and for which the information is + available. + + Note that an empty list is a possible valid response; for a + newsgroup-based keyword, it indicates that there are no groups + meeting the above criteria. + + If the keyword is not recognised, or if an argument is specified and + the keyword does not expect one, a 501 response code MUST BE + returned. If the keyword is recognised but the server does not + maintain the information, a 503 response code MUST BE returned. + + + + + +Feather Standards Track [Page 67] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The LIST command MUST NOT change the visible state of the server in + any way; that is, the behaviour of subsequent commands MUST NOT be + affected by whether the LIST command was issued. For example, it + MUST NOT make groups available that otherwise would not have been. + +7.6.1.3. Examples + + Example of LIST with the ACTIVE keyword: + + [C] LIST ACTIVE + [S] 215 list of newsgroups follows + [S] misc.test 3002322 3000234 y + [S] comp.risks 442001 441099 m + [S] alt.rfc-writers.recovery 4 1 y + [S] tx.natives.recovery 89 56 y + [S] tx.natives.recovery.d 11 9 n + [S] . + + Example of LIST with no keyword: + + [C] LIST + [S] 215 list of newsgroups follows + [S] misc.test 3002322 3000234 y + [S] comp.risks 442001 441099 m + [S] alt.rfc-writers.recovery 4 1 y + [S] tx.natives.recovery 89 56 y + [S] tx.natives.recovery.d 11 9 n + [S] . + + The output is identical to that of the previous example. + + Example of LIST on a newsgroup-based keyword with and without + wildmat: + + [C] LIST ACTIVE.TIMES + [S] 215 information follows + [S] misc.test 930445408 + [S] alt.rfc-writers.recovery 930562309 + [S] tx.natives.recovery 930678923 + [S] . + [C] LIST ACTIVE.TIMES tx.* + [S] 215 information follows + [S] tx.natives.recovery 930678923 + [S] . + + + + + + + +Feather Standards Track [Page 68] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of LIST returning an error where the keyword is recognized + but the software does not maintain this information: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] LIST ACTIVE NEWSGROUPS ACTIVE.TIMES XTRA.DATA + [S] . + [C] LIST XTRA.DATA + [S] 503 Data item not stored + + Example of LIST where the keyword is not recognised: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] LIST ACTIVE NEWSGROUPS ACTIVE.TIMES XTRA.DATA + [S] . + [C] LIST DISTRIB.PATS + [S] 501 Syntax Error + +7.6.2. Standard LIST Keywords + + This specification defines the following LIST keywords: + + +--------------+---------------+------------------------------------+ + | Keyword | Definition | Status | + +--------------+---------------+------------------------------------+ + | ACTIVE | Section 7.6.3 | Mandatory if the READER capability | + | | | is advertised | + | | | | + | ACTIVE.TIMES | Section 7.6.4 | Optional | + | | | | + | DISTRIB.PATS | Section 7.6.5 | Optional | + | | | | + | HEADERS | Section 8.6 | Mandatory if the HDR capability is | + | | | advertised | + | | | | + | NEWSGROUPS | Section 7.6.6 | Mandatory if the READER capability | + | | | is advertised | + | | | | + | OVERVIEW.FMT | Section 8.4 | Mandatory if the OVER capability | + | | | is advertised | + +--------------+---------------+------------------------------------+ + + + + + +Feather Standards Track [Page 69] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Where one of these LIST keywords is supported by a server, it MUST + have the meaning given in the relevant sub-section. + +7.6.3. LIST ACTIVE + + This keyword MUST be supported by servers advertising the READER + capability. + + LIST ACTIVE returns a list of valid newsgroups and associated + information. If no wildmat is specified, the server MUST include + every group that the client is permitted to select with the GROUP + command (Section 6.1.1). Each line of this list consists of four + fields separated from each other by one or more spaces: + + o The name of the newsgroup. + o The reported high water mark for the group. + o The reported low water mark for the group. + o The current status of the group on this server. + + The reported high and low water marks are as described in the GROUP + command (see Section 6.1.1), but note that they are in the opposite + order to the 211 response to that command. + + The status field is typically one of the following: + + "y" Posting is permitted. + + "n" Posting is not permitted. + + "m" Postings will be forwarded to the newsgroup moderator. + + The server SHOULD use these values when these meanings are required + and MUST NOT use them with any other meaning. Other values for the + status may exist; the definition of these other values and the + circumstances under which they are returned may be specified in an + extension or may be private to the server. A client SHOULD treat an + unrecognized status as giving no information. + + The status of a newsgroup only indicates how posts to that newsgroup + are normally processed and is not necessarily customised to the + specific client. For example, if the current client is forbidden + from posting, then this will apply equally to groups with status "y". + Conversely, a client with special privileges (not defined by this + specification) might be able to post to a group with status "n". + + + + + + + +Feather Standards Track [Page 70] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + For example: + + [C] LIST ACTIVE + [S] 215 list of newsgroups follows + [S] misc.test 3002322 3000234 y + [S] comp.risks 442001 441099 m + [S] alt.rfc-writers.recovery 4 1 y + [S] tx.natives.recovery 89 56 y + [S] tx.natives.recovery.d 11 9 n + [S] . + + or, on an implementation that includes leading zeroes: + + [C] LIST ACTIVE + [S] 215 list of newsgroups follows + [S] misc.test 0003002322 0003000234 y + [S] comp.risks 0000442001 0000441099 m + [S] alt.rfc-writers.recovery 0000000004 0000000001 y + [S] tx.natives.recovery 0000000089 0000000056 y + [S] tx.natives.recovery.d 0000000011 0000000009 n + [S] . + + The information is newsgroup based, and a wildmat MAY be specified, + in which case the response is limited to only the groups (if any) + whose names match the wildmat. For example: + + [C] LIST ACTIVE *.recovery + [S] 215 list of newsgroups follows + [S] alt.rfc-writers.recovery 4 1 y + [S] tx.natives.recovery 89 56 y + [S] . + +7.6.4. LIST ACTIVE.TIMES + + This keyword is optional. + + The active.times list is maintained by some NNTP servers to contain + information about who created a particular newsgroup and when. Each + line of this list consists of three fields separated from each other + by one or more spaces. The first field is the name of the newsgroup. + The second is the time when this group was created on this news + server, measured in seconds since the start of January 1, 1970. The + third is plain text intended to describe the entity that created the + newsgroup; it is often a mailbox as defined in RFC 2822 [RFC2822]. + For example: + + + + + + +Feather Standards Track [Page 71] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + [C] LIST ACTIVE.TIMES + [S] 215 information follows + [S] misc.test 930445408 + [S] alt.rfc-writers.recovery 930562309 + [S] tx.natives.recovery 930678923 + [S] . + + The list MAY omit newsgroups for which the information is unavailable + and MAY include groups not available on the server; in particular, it + MAY omit all groups created before the date and time of the oldest + entry. The client MUST NOT assume that the list is complete or that + it matches the list returned by the LIST ACTIVE command + (Section 7.6.3). The NEWGROUPS command (Section 7.3) may provide a + better way to access this information, and the results of the two + commands SHOULD be consistent except that, if the latter is invoked + with a date and time earlier than the oldest entry in active.times + list, its result may include extra groups. + + The information is newsgroup based, and a wildmat MAY be specified, + in which case the response is limited to only the groups (if any) + whose names match the wildmat. + +7.6.5. LIST DISTRIB.PATS + + This keyword is optional. + + The distrib.pats list is maintained by some NNTP servers to assist + clients to choose a value for the content of the Distribution header + of a news article being posted. Each line of this list consists of + three fields separated from each other by a colon (":"). The first + field is a weight, the second field is a wildmat (which may be a + simple newsgroup name), and the third field is a value for the + Distribution header content. For example: + + [C] LIST DISTRIB.PATS + [S] 215 information follows + [S] 10:local.*:local + [S] 5:*:world + [S] 20:local.here.*:thissite + [S] . + + The client MAY use this information to construct an appropriate + Distribution header given the name of a newsgroup. To do so, it + should determine the lines whose second field matches the newsgroup + name, select from among them the line with the highest weight (with 0 + being the lowest), and use the value of the third field to construct + the Distribution header. + + + + +Feather Standards Track [Page 72] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + The information is not newsgroup based, and an argument MUST NOT be + specified. + +7.6.6. LIST NEWSGROUPS + + This keyword MUST be supported by servers advertising the READER + capability. + + The newsgroups list is maintained by NNTP servers to contain the name + of each newsgroup that is available on the server and a short + description about the purpose of the group. Each line of this list + consists of two fields separated from each other by one or more space + or TAB characters (the usual practice is a single TAB). The first + field is the name of the newsgroup, and the second is a short + description of the group. For example: + + [C] LIST NEWSGROUPS + [S] 215 information follows + [S] misc.test General Usenet testing + [S] alt.rfc-writers.recovery RFC Writers Recovery + [S] tx.natives.recovery Texas Natives Recovery + [S] . + + The list MAY omit newsgroups for which the information is unavailable + and MAY include groups not available on the server. The client MUST + NOT assume that the list is complete or that it matches the list + returned by LIST ACTIVE. + + The description SHOULD be in UTF-8. However, servers often obtain + the information from external sources. These sources may have used + different encodings (ones that use octets in the range 128 to 255 in + some other manner) and, in that case, the server MAY pass it on + unchanged. Therefore, clients MUST be prepared to receive such + descriptions. + + The information is newsgroup based, and a wildmat MAY be specified, + in which case the response is limited to only the groups (if any) + whose names match the wildmat. + +8. Article Field Access Commands + + This section lists commands that may be used to access specific + article fields; that is, headers of articles and metadata about + articles. These commands typically fetch data from an "overview + database", which is a database of headers extracted from incoming + articles plus metadata determined as the article arrives. Only + certain fields are included in the database. + + + + +Feather Standards Track [Page 73] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + This section is based on the Overview/NOV database [ROBE1995] + developed by Geoff Collyer. + +8.1. Article Metadata + + Article "metadata" is data about articles that does not occur within + the article itself. Each metadata item has a name that MUST begin + with a colon (and that MUST NOT contain a colon elsewhere within it). + As with header names, metadata item names are not case sensitive. + + When generating a metadata item, the server MUST compute it for + itself and MUST NOT trust any related value provided in the article. + (In particular, a Lines or Bytes header in the article MUST NOT be + assumed to specify the correct number of lines or bytes in the + article.) If the server has access to several non-identical copies + of an article, the value returned MUST be correct for any copy of + that article retrieved during the same session. + + This specification defines two metadata items: ":bytes" and ":lines". + Other metadata items may be defined by extensions. The names of + metadata items defined by registered extensions MUST NOT begin with + ":x-". To avoid the risk of a clash with a future registered + extension, the names of metadata items defined by private extensions + SHOULD begin with ":x-". + +8.1.1. The :bytes Metadata Item + + The :bytes metadata item for an article is a decimal integer. It + SHOULD equal the number of octets in the entire article: headers, + body, and separating empty line (counting a CRLF pair as two octets, + and excluding both the "." CRLF terminating the response and any "." + added for "dot-stuffing" purposes). + + Note to client implementers: some existing servers return a value + different from that above. The commonest reasons for this are as + follows: + + o Counting a CRLF pair as one octet. + + o Including the "." character used for dot-stuffing in the number. + + o Including the terminating "." CRLF in the number. + + o Using one copy of an article for counting the octets but then + returning another one that differs in some (permitted) manner. + + Implementations should be prepared for such variation and MUST NOT + rely on the value being accurate. + + + +Feather Standards Track [Page 74] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +8.1.2. The :lines Metadata Item + + The :lines metadata item for an article is a decimal integer. It + MUST equal the number of lines in the article body (excluding the + empty line separating headers and body). Equivalently, it is two + less than the number of CRLF pairs that the BODY command would return + for that article (the extra two are those following the response code + and the termination octet). + +8.2. Database Consistency + + The information stored in the overview database may change over time. + If the database records the content or absence of a given field (that + is, a header or metadata item) for all articles, it is said to be + "consistent" for that field. If it records the content of a header + for some articles but not for others that nevertheless included that + header, or if it records a metadata item for some articles but not + for others to which that item applies, it is said to be + "inconsistent" for that field. + + The LIST OVERVIEW.FMT command SHOULD list all the fields for which + the database is consistent at that moment. It MAY omit such fields + (for example, if it is not known whether the database is consistent + or inconsistent). It MUST NOT include fields for which the database + is inconsistent or that are not stored in the database. Therefore, + if a header appears in the LIST OVERVIEW.FMT output but not in the + OVER output for a given article, that header does not appear in the + article (similarly for metadata items). + + These rules assume that the fields being stored in the database + remain constant for long periods of time, and therefore the database + will be consistent. When the set of fields to be stored is changed, + it will be inconsistent until either the database is rebuilt or the + only articles remaining are those received since the change. + Therefore, the output from LIST OVERVIEW.FMT needs to be altered + twice. Firstly, before any fields stop being stored they MUST be + removed from the output; then, when the database is once more known + to be consistent, the new fields SHOULD be added to the output. + + If the HDR command uses the overview database rather than taking + information directly from the articles, the same issues of + consistency and inconsistency apply, and the LIST HEADERS command + SHOULD take the same approach as the LIST OVERVIEW.FMT command in + resolving them. + + + + + + + +Feather Standards Track [Page 75] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +8.3. OVER + +8.3.1. Usage + + Indicating capability: OVER + + Syntax + OVER message-id + OVER range + OVER + + Responses + + First form (message-id specified) + 224 Overview information follows (multi-line) + 430 No article with that message-id + + Second form (range specified) + 224 Overview information follows (multi-line) + 412 No newsgroup selected + 423 No articles in that range + + Third form (current article number used) + 224 Overview information follows (multi-line) + 412 No newsgroup selected + 420 Current article number is invalid + + Parameters + range Number(s) of articles + message-id Message-id of article + +8.3.2. Description + + The OVER command returns the contents of all the fields in the + database for an article specified by message-id, or from a specified + article or range of articles in the currently selected newsgroup. + + The message-id argument indicates a specific article. The range + argument may be any of the following: + + o An article number. + + o An article number followed by a dash to indicate all following. + + o An article number followed by a dash followed by another article + number. + + If neither is specified, the current article number is used. + + + +Feather Standards Track [Page 76] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Support for the first (message-id) form is optional. If it is + supported, the OVER capability line MUST include the argument + "MSGID". Otherwise, the capability line MUST NOT include this + argument, and the OVER command MUST return the generic response code + 503 when this form is used. + + If the information is available, it is returned as a multi-line data + block following the 224 response code and contains one line per + article, sorted in numerical order of article number. (Note that + unless the argument is a range including a dash, there will be + exactly one line in the data block.) Each line consists of a number + of fields separated by a TAB. A field may be empty (in which case + there will be two adjacent TABs), and a sequence of trailing TABs may + be omitted. + + The first 8 fields MUST be the following, in order: + + "0" or article number (see below) + Subject header content + From header content + Date header content + Message-ID header content + References header content + :bytes metadata item + :lines metadata item + + If the article is specified by message-id (the first form of the + command), the article number MUST be replaced with zero, except that + if there is a currently selected newsgroup and the article is present + in that group, the server MAY use the article's number in that group. + (See the ARTICLE command (Section 6.2.1) and STAT examples + (Section 6.2.4.3) for more details.) In the other two forms of the + command, the article number MUST be returned. + + Any subsequent fields are the contents of the other headers and + metadata held in the database. + + For the five mandatory headers, the content of each field MUST be + based on the content of the header (that is, with the header name and + following colon and space removed). If the article does not contain + that header, or if the content is empty, the field MUST be empty. + For the two mandatory metadata items, the content of the field MUST + be just the value, with no other text. + + For all subsequent fields that contain headers, the content MUST be + the entire header line other than the trailing CRLF. For all + subsequent fields that contain metadata, the field consists of the + metadata name, a single space, and then the value. + + + +Feather Standards Track [Page 77] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + For all fields, the value is processed by first removing all CRLF + pairs (that is, undoing any folding and removing the terminating + CRLF) and then replacing each TAB with a single space. If there is + no such header in the article, no such metadata item, or no header or + item stored in the database for that article, the corresponding field + MUST be empty. + + Note that, after unfolding, the characters NUL, LF, and CR cannot + occur in the header of an article offered by a conformant server. + Nevertheless, servers SHOULD check for these characters and replace + each one by a single space (so that, for example, CR LF LF TAB will + become two spaces, since the CR and first LF will be removed by the + unfolding process). This will encourage robustness in the face of + non-conforming data; it is also possible that future versions of this + specification could permit these characters to appear in articles. + + The server SHOULD NOT produce output for articles that no longer + exist. + + If the argument is a message-id and no such article exists, a 430 + response MUST be returned. If the argument is a range or is omitted + and the currently selected newsgroup is invalid, a 412 response MUST + be returned. If the argument is a range and no articles in that + number range exist in the currently selected newsgroup, including the + case where the second number is less than the first one, a 423 + response MUST be returned. If the argument is omitted and the + current article number is invalid, a 420 response MUST be returned. + +8.3.3. Examples + + In the first four examples, TAB has been replaced by vertical bar and + some lines have been folded for readability. + + Example of a successful retrieval of overview information for an + article (explicitly not using an article number): + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] OVER + [S] 224 Overview information follows + [S] 3000234|I am just a test article|"Demo User" + |6 Oct 1998 04:38:40 -0500| + <45223423@example.com>|<45454@example.net>|1234| + 17|Xref: news.example.com misc.test:3000363 + [S] . + + + + + + +Feather Standards Track [Page 78] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of a successful retrieval of overview information for an + article by message-id: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] OVER MSGID + [S] LIST ACTIVE NEWSGROUPS OVERVIEW.FMT + [S] . + [C] OVER <45223423@example.com> + [S] 224 Overview information follows + [S] 0|I am just a test article|"Demo User" + |6 Oct 1998 04:38:40 -0500| + <45223423@example.com>|<45454@example.net>|1234| + 17|Xref: news.example.com misc.test:3000363 + [S] . + + Note that the article number has been replaced by "0". + + Example of the same commands on a system that does not implement + retrieval by message-id: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] OVER + [S] LIST ACTIVE NEWSGROUPS OVERVIEW.FMT + [S] . + [C] OVER <45223423@example.com> + [S] 503 Overview by message-id unsupported + + + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 79] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of a successful retrieval of overview information for a range + of articles: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] OVER 3000234-3000240 + [S] 224 Overview information follows + [S] 3000234|I am just a test article|"Demo User" + |6 Oct 1998 04:38:40 -0500| + <45223423@example.com>|<45454@example.net>|1234| + 17|Xref: news.example.com misc.test:3000363 + [S] 3000235|Another test article|nobody@nowhere.to + (Demo User)|6 Oct 1998 04:38:45 -0500|<45223425@to.to>|| + 4818|37||Distribution: fi + [S] 3000238|Re: I am just a test article|somebody@elsewhere.to| + 7 Oct 1998 11:38:40 +1200|| + <45223423@to.to>|9234|51 + [S] . + + Note the missing "References" and Xref headers in the second line, + the missing trailing fields in the first and last lines, and that + there are only results for those articles that still exist. + + Example of an unsuccessful retrieval of overview information on an + article by number: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] OVER 300256 + [S] 423 No such article in this group + + Example of an invalid range: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] OVER 3000444-3000222 + [S] 423 Empty range + + Example of an unsuccessful retrieval of overview information by + number because no newsgroup was selected first: + + [Assumes currently selected newsgroup is invalid.] + [C] OVER + [S] 412 No newsgroup selected + + + + + + + +Feather Standards Track [Page 80] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an attempt to retrieve information when the currently + selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] OVER + [S] 420 No current article selected + +8.4. LIST OVERVIEW.FMT + +8.4.1. Usage + + Indicating capability: OVER + + Syntax + LIST OVERVIEW.FMT + + Responses + 215 Information follows (multi-line) + +8.4.2. Description + + See Section 7.6.1 for general requirements of the LIST command. + + The LIST OVERVIEW.FMT command returns a description of the fields in + the database for which it is consistent (as described above). The + information is returned as a multi-line data block following the 215 + response code. The information contains one line per field in the + order in which they are returned by the OVER command; the first 7 + lines MUST (except for the case of letters) be exactly as follows: + + Subject: + From: + Date: + Message-ID: + References: + :bytes + :lines + + For compatibility with existing implementations, the last two lines + MAY instead be: + + Bytes: + Lines: + + even though they refer to metadata, not headers. + + + + + +Feather Standards Track [Page 81] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + All subsequent lines MUST consist of either a header name followed by + ":full", or the name of a piece of metadata. + + There are no leading or trailing spaces in the output. + + Note that the 7 fixed lines describe the 2nd to 8th fields of the + OVER output. The "full" suffix (which may use either uppercase, + lowercase, or a mix) is a reminder that the corresponding fields + include the header name. + + This command MAY generate different results if it is used more than + once in a session. + + If the OVER command is not implemented, the meaning of the output + from this command is not specified, but it must still meet the above + syntactic requirements. + +8.4.3. Examples + + Example of LIST OVERVIEW.FMT output corresponding to the example OVER + output above, in the preferred format: + + [C] LIST OVERVIEW.FMT + [S] 215 Order of fields in overview database. + [S] Subject: + [S] From: + [S] Date: + [S] Message-ID: + [S] References: + [S] :bytes + [S] :lines + [S] Xref:full + [S] Distribution:full + [S] . + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 82] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of LIST OVERVIEW.FMT output corresponding to the example OVER + output above, in the alternative format: + + [C] LIST OVERVIEW.FMT + [S] 215 Order of fields in overview database. + [S] Subject: + [S] From: + [S] Date: + [S] Message-ID: + [S] References: + [S] Bytes: + [S] Lines: + [S] Xref:FULL + [S] Distribution:FULL + [S] . + +8.5. HDR + +8.5.1. Usage + + Indicating capability: HDR + + Syntax + HDR field message-id + HDR field range + HDR field + + Responses + + First form (message-id specified) + 225 Headers follow (multi-line) + 430 No article with that message-id + + Second form (range specified) + 225 Headers follow (multi-line) + 412 No newsgroup selected + 423 No articles in that range + + Third form (current article number used) + 225 Headers follow (multi-line) + 412 No newsgroup selected + 420 Current article number is invalid + + Parameters + field Name of field + range Number(s) of articles + message-id Message-id of article + + + + +Feather Standards Track [Page 83] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +8.5.2. Description + + The HDR command provides access to specific fields from an article + specified by message-id, or from a specified article or range of + articles in the currently selected newsgroup. It MAY take the + information directly from the articles or from the overview database. + In the case of headers, an implementation MAY restrict the use of + this command to a specific list of headers or MAY allow it to be used + with any header; it may behave differently when it is used with a + message-id argument and when it is used with a range or no argument. + + The required field argument is the name of a header with the colon + omitted (e.g., "subject") or the name of a metadata item including + the leading colon (e.g., ":bytes"), and is case insensitive. + + The message-id argument indicates a specific article. The range + argument may be any of the following: + + o An article number. + + o An article number followed by a dash to indicate all following. + + o An article number followed by a dash followed by another article + number. + + If neither is specified, the current article number is used. + + If the information is available, it is returned as a multi-line data + block following the 225 response code and contains one line for each + article in the range that exists. (Note that unless the argument is + a range including a dash, there will be exactly one line in the data + block.) The line consists of the article number, a space, and then + the contents of the field. In the case of a header, the header name, + the colon, and the first space after the colon are all omitted. + + If the article is specified by message-id (the first form of the + command), the article number MUST be replaced with zero, except that + if there is a currently selected newsgroup and the article is present + in that group, the server MAY use the article's number in that group. + (See the ARTICLE command (Section 6.2.1) and STAT examples + (Section 6.2.4.3) for more details.) In the other two forms of the + command, the article number MUST be returned. + + Header contents are modified as follows: all CRLF pairs are removed, + and then each TAB is replaced with a single space. (Note that this + is the same transformation as is performed by the OVER command + (Section 8.3.2), and the same comment concerning NUL, CR, and LF + applies.) + + + +Feather Standards Track [Page 84] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Note the distinction between headers and metadata appearing to have + the same meaning. Headers are always taken unchanged from the + article; metadata are always calculated. For example, a request for + "Lines" returns the contents of the "Lines" header of the specified + articles, if any, no matter whether they accurately state the number + of lines, while a request for ":lines" returns the line count + metadata, which is always the actual number of lines irrespective of + what any header may state. + + If the requested header is not present in the article, or if it is + present but empty, a line for that article is included in the output, + but the header content portion of the line is empty (the space after + the article number MAY be retained or omitted). If the header occurs + in a given article more than once, only the content of the first + occurrence is returned by HDR. If any article number in the provided + range does not exist in the group, no line for that article number is + included in the output. + + If the second argument is a message-id and no such article exists, a + 430 response MUST be returned. If the second argument is a range or + is omitted and the currently selected newsgroup is invalid, a 412 + response MUST be returned. If the second argument is a range and no + articles in that number range exist in the currently selected + newsgroup, including the case where the second number is less than + the first one, a 423 response MUST be returned. If the second + argument is omitted and the current article number is invalid, a 420 + response MUST be returned. + + A server MAY only allow HDR commands for a limited set of fields; it + may behave differently in this respect for the first (message-id) + form from how it would for the other forms. If so, it MUST respond + with the generic 503 response to attempts to request other fields, + rather than return erroneous results, such as a successful empty + response. + + If HDR uses the overview database and it is inconsistent for the + requested field, the server MAY return what results it can, or it MAY + respond with the generic 503 response. In the latter case, the field + MUST NOT appear in the output from LIST HEADERS. + + + + + + + + + + + + +Feather Standards Track [Page 85] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +8.5.3. Examples + + Example of a successful retrieval of subject lines from a range of + articles (3000235 has no Subject header, and 3000236 is missing): + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HDR Subject 3000234-3000238 + [S] 225 Headers follow + [S] 3000234 I am just a test article + [S] 3000235 + [S] 3000237 Re: I am just a test article + [S] 3000238 Ditto + [S] . + + Example of a successful retrieval of line counts from a range of + articles: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HDR :lines 3000234-3000238 + [S] 225 Headers follow + [S] 3000234 42 + [S] 3000235 5 + [S] 3000237 11 + [S] 3000238 2378 + [S] . + + Example of a successful retrieval of the subject line from an article + by message-id: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HDR subject + [S] 225 Header information follows + [S] 0 I am just a test article + [S] . + + Example of a successful retrieval of the subject line from the + current article: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HDR subject + [S] 225 Header information follows + [S] 3000234 I am just a test article + [S] . + + + + +Feather Standards Track [Page 86] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an unsuccessful retrieval of a header from an article by + message-id: + + [C] HDR subject + [S] 430 No Such Article Found + + Example of an unsuccessful retrieval of headers from articles by + number because no newsgroup was selected first: + + [Assumes currently selected newsgroup is invalid.] + [C] HDR subject 300256- + [S] 412 No newsgroup selected + + Example of an unsuccessful retrieval of headers because the currently + selected newsgroup is empty: + + [C] GROUP example.empty.newsgroup + [S] 211 0 0 0 example.empty.newsgroup + [C] HDR subject 1- + [S] 423 No articles in that range + + Example of an unsuccessful retrieval of headers because the server + does not allow HDR commands for that header: + + [C] GROUP misc.test + [S] 211 1234 3000234 3002322 misc.test + [C] HDR Content-Type 3000234-3000238 + [S] 503 HDR not permitted on Content-Type + +8.6. LIST HEADERS + +8.6.1. Usage + + Indicating capability: HDR + + Syntax + LIST HEADERS [MSGID|RANGE] + + Responses + 215 Field list follows (multi-line) + + Parameters + MSGID Requests list for access by message-id + RANGE Requests list for access by range + + + + + + + +Feather Standards Track [Page 87] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +8.6.2. Description + + See Section 7.6.1 for general requirements of the LIST command. + + The LIST HEADERS command returns a list of fields that may be + retrieved using the HDR command. + + The information is returned as a multi-line data block following the + 215 response code and contains one line for each field name + (excluding the trailing colon for headers and including the leading + colon for metadata items). If the implementation allows any header + to be retrieved, it MUST NOT include any header names in the list but + MUST include the special entry ":" (a single colon on its own). It + MUST still explicitly list any metadata items that are available. + The order of items in the list is not significant; the server need + not even consistently return the same order. The list MAY be empty + (though in this circumstance there is little point in providing the + HDR command). + + An implementation that also supports the OVER command SHOULD at least + permit all the headers and metadata items listed in the output from + the LIST OVERVIEW.FMT command. + + If the server treats the first form of the HDR command (message-id + specified) differently from the other two forms (range specified or + current article number used) in respect of which headers or metadata + items are available, then the following apply: + + o If the MSGID argument is specified, the results MUST be those + available for the first form of the HDR command. + + o If the RANGE argument is specified, the results MUST be those + available for the second and third forms of the HDR command. + + o If no argument is specified, the results MUST be those available + in all forms of the HDR command (that is, it MUST only list those + items listed in both the previous cases). + + If the server does not treat the various forms differently, then it + MUST ignore any argument and always produce the same results (though + not necessarily always in the same order). + + If the HDR command is not implemented, the meaning of the output from + this command is not specified, but it must still meet the above + syntactic requirements. + + + + + + +Feather Standards Track [Page 88] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +8.6.3. Examples + + Example of an implementation providing access to only a few headers: + + [C] LIST HEADERS + [S] 215 headers supported: + [S] Subject + [S] Message-ID + [S] Xref + [S] . + + Example of an implementation providing access to the same fields as + the first example in Section 8.4.3: + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] OVER + [S] HDR + [S] LIST ACTIVE NEWSGROUPS HEADERS OVERVIEW.FMT + [S] . + [C] LIST HEADERS + [S] 215 headers and metadata items supported: + [S] Date + [S] Distribution + [S] From + [S] Message-ID + [S] References + [S] Subject + [S] Xref + [S] :bytes + [S] :lines + [S] . + + Example of an implementation providing access to all headers: + + [C] LIST HEADERS + [S] 215 metadata items supported: + [S] : + [S] :lines + [S] :bytes + [S] :x-article-number + [S] . + + + + + + + +Feather Standards Track [Page 89] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Example of an implementation distinguishing the first form of the HDR + command from the other two forms: + + [C] LIST HEADERS RANGE + [S] 215 metadata items supported: + [S] : + [S] :lines + [S] :bytes + [S] . + [C] LIST HEADERS MSGID + [S] 215 headers and metadata items supported: + [S] Date + [S] Distribution + [S] From + [S] Message-ID + [S] References + [S] Subject + [S] :lines + [S] :bytes + [S] :x-article-number + [S] . + [C] LIST HEADERS + [S] 215 headers and metadata items supported: + [S] Date + [S] Distribution + [S] From + [S] Message-ID + [S] References + [S] Subject + [S] :lines + [S] :bytes + [S] . + + Note that :x-article-number does not appear in the last set of + output. + +9. Augmented BNF Syntax for NNTP + +9.1. Introduction + + Each of the following sections describes the syntax of a major + element of NNTP. This syntax extends and refines the descriptions + elsewhere in this specification and should be given precedence when + resolving apparent conflicts. Note that ABNF [RFC4234] strings are + case insensitive. Non-terminals used in several places are defined + in a separate section at the end. + + + + + +Feather Standards Track [Page 90] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Between them, the non-terminals , , + , and specify the text that flows + between client and server. A consistent naming scheme is used in + this document for the non-terminals relating to each command, and + SHOULD be used by the specification of registered extensions. + + For each command, the sequence is as follows: + + o The client sends an instance of ; the syntax for the + EXAMPLE command is . + + o If the client is one that immediately streams data, it sends an + instance of ; the syntax for the EXAMPLE + command is . + + o The server sends an instance of . + + * The initial response line is independent of the command that + generated it; if the 000 response has arguments, the syntax of + the initial line is . + + * If the response is multi-line, the initial line is followed by + a . The syntax for the contents of this + block after "dot-stuffing" has been removed is (for the 000 + response to the EXAMPLE command) and + is an instance of . + + o While the latest response is one that indicates more data is + required (in general, a 3xx response): + + * the client sends an instance of ; the + syntax for the EXAMPLE continuation following a 333 response is + ; + + * the server sends another instance of , as above. + + (There are no commands in this specification that immediately stream + data, but this non-terminal is defined for the convenience of + extensions.) + + + + + + + + + + + + +Feather Standards Track [Page 91] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +9.2. Commands + + This syntax defines the non-terminal , which represents + what is sent from the client to the server (see section 3.1 for + limits on lengths). + + command-line = command EOL + command = X-command + X-command = keyword *(WS token) + + command =/ article-command / + body-command / + capabilities-command / + date-command / + group-command / + hdr-command / + head-command / + help-command / + ihave-command / + last-command / + list-command / + listgroup-command / + mode-reader-command / + newgroups-command / + newnews-command / + next-command / + over-command / + post-command / + quit-command / + stat-command + + article-command = "ARTICLE" [WS article-ref] + body-command = "BODY" [WS article-ref] + capabilities-command = "CAPABILITIES" [WS keyword] + date-command = "DATE" + group-command = "GROUP" [WS newsgroup-name] + hdr-command = "HDR" WS header-meta-name [WS range-ref] + head-command = "HEAD" [WS article-ref] + help-command = "HELP" + ihave-command = "IHAVE" WS message-id + last-command = "LAST" + list-command = "LIST" [WS list-arguments] + listgroup-command = "LISTGROUP" [WS newsgroup-name [WS range]] + mode-reader-command = "MODE" WS "READER" + newgroups-command = "NEWGROUPS" WS date-time + newnews-command = "NEWNEWS" WS wildmat WS date-time + next-command = "NEXT" + over-command = "OVER" [WS range-ref] + + + +Feather Standards Track [Page 92] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + post-command = "POST" + quit-command = "QUIT" + stat-command = "STAT" [WS article-ref] + + article-ref = article-number / message-id + date = date2y / date4y + date4y = 4DIGIT 2DIGIT 2DIGIT + date2y = 2DIGIT 2DIGIT 2DIGIT + date-time = date WS time [WS "GMT"] + header-meta-name = header-name / metadata-name + list-arguments = keyword [WS token] + metadata-name = ":" 1*A-NOTCOLON + range = article-number ["-" [article-number]] + range-ref = range / message-id + time = 2DIGIT 2DIGIT 2DIGIT + +9.3. Command Continuation + + This syntax defines the further material sent by the client in the + case of multi-stage commands and those that stream data. + + command-datastream = UNDEFINED + ; not used, provided as a hook for extensions + command-continuation = ihave-335-continuation / + post-340-continuation + + ihave-335-continuation = encoded-article + post-340-continuation = encoded-article + + encoded-article = multi-line-data-block + ; after undoing the "dot-stuffing", this MUST match
+ +9.4. Responses + +9.4.1. Generic Responses + + This syntax defines the non-terminal , which represents the + generic form of responses; that is, what is sent from the server to + the client in response to a or a . + + response = simple-response / multi-line-response + simple-response = initial-response-line + multi-line-response = initial-response-line multi-line-data-block + + initial-response-line = + initial-response-content [SP trailing-comment] CRLF + initial-response-content = X-initial-response-content + X-initial-response-content = 3DIGIT *(SP response-argument) + + + +Feather Standards Track [Page 93] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + response-argument = 1*A-CHAR + trailing-comment = *U-CHAR + +9.4.2. Initial Response Line Contents + + This syntax defines the specific initial response lines for the + various commands in this specification (see section 3.1 for limits on + lengths). Only those response codes with arguments are listed. + + initial-response-content =/ response-111-content / + response-211-content / + response-220-content / + response-221-content / + response-222-content / + response-223-content / + response-401-content + + response-111-content = "111" SP date4y time + response-211-content = "211" 3(SP article-number) SP newsgroup-name + response-220-content = "220" SP article-number SP message-id + response-221-content = "221" SP article-number SP message-id + response-222-content = "222" SP article-number SP message-id + response-223-content = "223" SP article-number SP message-id + response-401-content = "401" SP capability-label + +9.4.3. Multi-line Response Contents + + This syntax defines the content of the various multi-line responses; + more precisely, it defines the part of the response in the multi-line + data block after any "dot-stuffing" has been undone. The numeric + portion of each non-terminal name indicates the response code that is + followed by this data. + + multi-line-response-content = article-220-ml-content / + body-222-ml-content / + capabilities-101-ml-content / + hdr-225-ml-content / + head-221-ml-content / + help-100-ml-content / + list-215-ml-content / + listgroup-211-ml-content / + newgroups-231-ml-content / + newnews-230-ml-content / + over-224-ml-content + + article-220-ml-content = article + body-222-ml-content = body + capabilities-101-ml-content = version-line CRLF + + + +Feather Standards Track [Page 94] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + *(capability-line CRLF) + hdr-225-ml-content = *(article-number SP hdr-content CRLF) + head-221-ml-content = 1*header + help-100-ml-content = *(*U-CHAR CRLF) + list-215-ml-content = list-content + listgroup-211-ml-content = *(article-number CRLF) + newgroups-231-ml-content = active-groups-list + newnews-230-ml-content = *(message-id CRLF) + over-224-ml-content = *(article-number over-content CRLF) + + active-groups-list = *(newsgroup-name SPA article-number + SPA article-number SPA newsgroup-status CRLF) + hdr-content = *S-NONTAB + hdr-n-content = [(header-name ":" / metadata-name) SP hdr-content] + list-content = body + newsgroup-status = %x79 / %x6E / %x6D / private-status + over-content = 1*6(TAB hdr-content) / + 7(TAB hdr-content) *(TAB hdr-n-content) + private-status = token ; except the values in newsgroup-status + +9.5. Capability Lines + + This syntax defines the generic form of a capability line in the + capabilities list (see Section 3.3.1). + + capability-line = capability-entry + capability-entry = X-capability-entry + X-capability-entry = capability-label *(WS capability-argument) + capability-label = keyword + capability-argument = token + + This syntax defines the specific capability entries for the + capabilities in this specification. + + capability-entry =/ + hdr-capability / + ihave-capability / + implementation-capability / + list-capability / + mode-reader-capability / + newnews-capability / + over-capability / + post-capability / + reader-capability + + hdr-capability = "HDR" + ihave-capability = "IHAVE" + implementation-capability = "IMPLEMENTATION" *(WS token) + + + +Feather Standards Track [Page 95] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + list-capability = "LIST" 1*(WS keyword) + mode-reader-capability = "MODE-READER" + newnews-capability = "NEWNEWS" + over-capability = "OVER" [WS "MSGID"] + post-capability = "POST" + reader-capability = "READER" + + version-line = "VERSION" 1*(WS version-number) + version-number = nzDIGIT *5DIGIT + +9.6. LIST Variants + + This section defines more specifically the keywords for the LIST + command and the syntax of the corresponding response contents. + + ; active + list-arguments =/ "ACTIVE" [WS wildmat] + list-content =/ list-active-content + list-active-content = active-groups-list + + + ; active.times + list-arguments =/ "ACTIVE.TIMES" [WS wildmat] + list-content =/ list-active-times-content + list-active-times-content = + *(newsgroup-name SPA 1*DIGIT SPA newsgroup-creator CRLF) + newsgroup-creator = U-TEXT + + + ; distrib.pats + list-arguments =/ "DISTRIB.PATS" + list-content =/ list-distrib-pats-content + list-distrib-pats-content = + *(1*DIGIT ":" wildmat ":" distribution CRLF) + distribution = token + + + ; headers + list-arguments =/ "HEADERS" [WS ("MSGID" / "RANGE")] + list-content =/ list-headers-content + list-headers-content = *(header-meta-name CRLF) / + *((metadata-name / ":") CRLF) + + + ; newsgroups + list-arguments =/ "NEWSGROUPS" [WS wildmat] + list-content =/ list-newsgroups-content + list-newsgroups-content = + + + +Feather Standards Track [Page 96] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + *(newsgroup-name WS newsgroup-description CRLF) + newsgroup-description = S-TEXT + + + ; overview.fmt + list-arguments =/ "OVERVIEW.FMT" + list-content =/ list-overview-fmt-content + list-overview-fmt-content = "Subject:" CRLF + "From:" CRLF + "Date:" CRLF + "Message-ID:" CRLF + "References:" CRLF + ( ":bytes" CRLF ":lines" / "Bytes:" CRLF "Lines:") CRLF + *((header-name ":full" / metadata-name) CRLF) + +9.7. Articles + + This syntax defines the non-terminal
, which represents the + format of an article as described in Section 3.6. + + article = 1*header CRLF body + header = header-name ":" [CRLF] SP header-content CRLF + header-content = *(S-CHAR / [CRLF] WS) + body = *(*B-CHAR CRLF) + +9.8. General Non-terminals + + These non-terminals are used at various places in the syntax and are + collected here for convenience. A few of these non-terminals are not + used in this specification but are provided for the consistency and + convenience of extension authors. + + multi-line-data-block = content-lines termination + content-lines = *([content-text] CRLF) + content-text = (".." / B-NONDOT) *B-CHAR + termination = "." CRLF + + article-number = 1*16DIGIT + header-name = 1*A-NOTCOLON + keyword = ALPHA 2*(ALPHA / DIGIT / "." / "-") + message-id = "<" 1*248A-NOTGT ">" + newsgroup-name = 1*wildmat-exact + token = 1*P-CHAR + + wildmat = wildmat-pattern *("," ["!"] wildmat-pattern) + wildmat-pattern = 1*wildmat-item + wildmat-item = wildmat-exact / wildmat-wild + wildmat-exact = %x22-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E / + + + +Feather Standards Track [Page 97] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + UTF8-non-ascii ; exclude ! * , ? [ \ ] + wildmat-wild = "*" / "?" + + base64 = *(4base64-char) [base64-terminal] + base64-char = UPPER / LOWER / DIGIT / "+" / "/" + base64-terminal = 2base64-char "==" / 3base64-char "=" + + ; Assorted special character sets + ; A- means based on US-ASCII, excluding controls and SP + ; P- means based on UTF-8, excluding controls and SP + ; U- means based on UTF-8, excluding NUL CR and LF + ; B- means based on bytes, excluding NUL CR and LF + A-CHAR = %x21-7E + A-NOTCOLON = %x21-39 / %x3B-7E ; exclude ":" + A-NOTGT = %x21-3D / %x3F-7E ; exclude ">" + P-CHAR = A-CHAR / UTF8-non-ascii + U-CHAR = CTRL / TAB / SP / A-CHAR / UTF8-non-ascii + U-NONTAB = CTRL / SP / A-CHAR / UTF8-non-ascii + U-TEXT = P-CHAR *U-CHAR + B-CHAR = CTRL / TAB / SP / %x21-FF + B-NONDOT = CTRL / TAB / SP / %x21-2D / %x2F-FF ; exclude "." + + ALPHA = UPPER / LOWER ; use only when case-insensitive + CR = %x0D + CRLF = CR LF + CTRL = %x01-08 / %x0B-0C / %x0E-1F + DIGIT = %x30-39 + nzDIGIT = %x31-39 + EOL = *(SP / TAB) CRLF + LF = %x0A + LOWER = %x61-7A + SP = %x20 + SPA = 1*SP + TAB = %x09 + UPPER = %x41-5A + UTF8-non-ascii = UTF8-2 / UTF8-3 / UTF8-4 + UTF8-2 = %xC2-DF UTF8-tail + UTF8-3 = %xE0 %xA0-BF UTF8-tail / %xE1-EC 2UTF8-tail / + %xED %x80-9F UTF8-tail / %xEE-EF 2UTF8-tail + UTF8-4 = %xF0 %x90-BF 2UTF8-tail / %xF1-F3 3UTF8-tail / + %xF4 %x80-8F 2UTF8-tail + UTF8-tail = %x80-BF + WS = 1*(SP / TAB) + + The following non-terminals require special consideration. They + represent situations where material SHOULD be restricted to UTF-8, + but implementations MUST be able to cope with other character + encodings. Therefore, there are two sets of definitions for them. + + + +Feather Standards Track [Page 98] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Implementations MUST accept any content that meets this syntax: + + S-CHAR = %x21-FF + S-NONTAB = CTRL / SP / S-CHAR + S-TEXT = (CTRL / S-CHAR) *B-CHAR + + and MAY pass such content on unaltered. + + When generating new content or re-encoding existing content, + implementations SHOULD conform to this syntax: + + S-CHAR = P-CHAR + S-NONTAB = U-NONTAB + S-TEXT = U-TEXT + +9.9. Extensions and Validation + + The specification of a registered extension MUST include formal + syntax that defines additional forms for the following non-terminals: + + command + for each new command other than a variant of the LIST command - + the syntax of each command MUST be compatible with the definition + of ; + + command-datastream + for each new command that immediately streams data; + + command-continuation + for each new command that sends further material after the initial + command line - the syntax of each continuation MUST be exactly + what is sent to the server, including any escape mechanisms such + as "dot-stuffing"; + + initial-response-content + for each new response code that has arguments - the syntax of each + response MUST be compatible with the definition of ; + + multi-line-response-content + for each new response code that has a multi-line response - the + syntax MUST show the response after the lines containing the + response code and the terminating octet have been removed and any + "dot-stuffing" undone; + + capability-entry + for each new capability label - the syntax of each entry MUST be + compatible with the definition of ; + + + +Feather Standards Track [Page 99] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + list-arguments + for each new variant of the LIST command - the syntax of each + entry MUST be compatible with the definition of ; + + list-content + for each new variant of the LIST command - the syntax MUST show + the response after the lines containing the 215 response code and + the terminating octet have been removed and any "dot-stuffing" + undone. + + The =/ notation of ABNF [RFC4234] and the naming conventions + described in Section 9.1 SHOULD be used for this. + + When the syntax in this specification, or syntax based on it, is + validated, it should be noted that: + + o the non-terminals , , + , , and + describe basic concepts of the + protocol and are not referred to by any other rule; + + o the non-terminal is provided for the convenience of + extension authors and is not referred to by any rule in this + specification; + + o for the reasons given above, the non-terminals , + , and each have two definitions; and + + o the non-terminal is deliberately not defined. + +10. Internationalisation Considerations + +10.1. Introduction and Historical Situation + + RFC 977 [RFC977] was written at a time when internationalisation was + not seen as a significant issue. As such, it was written on the + assumption that all communication would be in ASCII and use only a + 7-bit transport layer, although in practice just about all known + implementations are 8-bit clean. + + Since then, Usenet and NNTP have spread throughout the world. In the + absence of standards for handling the issues of language and + character sets, countries, newsgroup hierarchies, and individuals + have found a variety of solutions that work for them but that are not + necessarily appropriate elsewhere. For example, some have adopted a + default 8-bit character set appropriate to their needs (such as + ISO/IEC 8859-1 in Western Europe or KOI-8 in Russia), others have + used ASCII (either US-ASCII or national variants) in headers but + + + +Feather Standards Track [Page 100] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + local 16-bit character sets in article bodies, and still others have + gone for a combination of MIME [RFC2045] and UTF-8. With the + increased use of MIME in email, it is becoming more common to find + NNTP articles containing MIME headers that identify the character set + of the body, but this is far from universal. + + The resulting confusion does not help interoperability. + + One point that has been generally accepted is that articles can + contain octets with the top bit set, and NNTP is only expected to + operate on 8-bit clean transport paths. + +10.2. This Specification + + Part of the role of this present specification is to eliminate this + confusion and promote interoperability as far as possible. At the + same time, it is necessary to accept the existence of the present + situation and not break existing implementations and arrangements + gratuitously, even if they are less than optimal. Therefore, the + current practice described above has been taken into consideration in + producing this specification. + + This specification extends NNTP from US-ASCII [ANSI1986] to UTF-8 + [RFC3629]. Except in the two areas discussed below, UTF-8 (which is + a superset of US-ASCII) is mandatory, and implementations MUST NOT + use any other encoding. + + Firstly, the use of MIME for article headers and bodies is strongly + recommended. However, given widely divergent existing practices, an + attempt to require a particular encoding and tagging standard would + be premature at this time. Accordingly, this specification allows + the use of arbitrary 8-bit data in articles subject to the following + requirements and recommendations. + + o The names of headers (e.g., "From" or "Subject") MUST be in + US-ASCII. + + o Header values SHOULD use US-ASCII or an encoding based on it, such + as RFC 2047 [RFC2047], until such time as another approach has + been standardised. At present, 8-bit encodings (including UTF-8) + SHOULD NOT be used because they are likely to cause + interoperability problems. + + o The character set of article bodies SHOULD be indicated in the + article headers, and this SHOULD be done in accordance with MIME. + + o Where an article is obtained from an external source, an + implementation MAY pass it on and derive data from it (such as the + + + +Feather Standards Track [Page 101] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + response to the HDR command), even though the article or the data + does not meet the above requirements. Implementations MUST + transfer such articles and data correctly and unchanged; they MUST + NOT attempt to convert or re-encode the article or derived data. + (Nevertheless, a client or server MAY elect not to post or forward + the article if, after further examination of the article, it deems + it inappropriate to do so.) + + This requirement affects the ARTICLE (Section 6.2.1), BODY + (Section 6.2.3), HDR (Section 8.5), HEAD (Section 6.2.2), IHAVE + (Section 6.3.2), OVER (Section 8.3), and POST (Section 6.3.1) + commands. + + Secondly, the following requirements are placed on the newsgroups + list returned by the LIST NEWSGROUPS command (Section 7.6.6): + + o Although this specification allows UTF-8 for newsgroup names, they + SHOULD be restricted to US-ASCII until a successor to RFC 1036 + [RFC1036] standardises another approach. 8-bit encodings SHOULD + NOT be used because they are likely to cause interoperability + problems. + + o The newsgroup description SHOULD be in US-ASCII or UTF-8 unless + and until a successor to RFC 1036 standardises other encoding + arrangements. 8-bit encodings other than UTF-8 SHOULD NOT be used + because they are likely to cause interoperability problems. + + o Implementations that obtain this data from an external source MUST + handle it correctly even if it does not meet the above + requirements. Implementations (in particular, clients) MUST + handle such data correctly. + +10.3. Outstanding Issues + + While the primary use of NNTP is for transmitting articles that + conform to RFC 1036 (Netnews articles), it is also used for other + formats (see Appendix A). It is therefore most appropriate that + internationalisation issues related to article formats be addressed + in the relevant specifications. For Netnews articles, this is any + successor to RFC 1036. For email messages, it is RFC 2822 [RFC2822]. + + Of course, any article transmitted via NNTP needs to conform to this + specification as well. + + Restricting newsgroup names to UTF-8 is not a complete solution. In + particular, when new newsgroup names are created or a user is asked + to enter a newsgroup name, some scheme of canonicalisation will need + to take place. This specification does not attempt to define that + + + +Feather Standards Track [Page 102] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + canonicalization; further work is needed in this area, in conjunction + with the article format specifications. Until such specifications + are published, implementations SHOULD match newsgroup names octet by + octet. It is anticipated that any approved scheme will be applied + "at the edges", and therefore octet-by-octet comparison will continue + to apply to most, if not all, uses of newsgroup names in NNTP. + + In the meantime, any implementation experimenting with UTF-8 + newsgroup names is strongly cautioned that a future specification may + require that those names be canonicalized when used with NNTP in a + way that is not compatible with their experiments. + + Since the primary use of NNTP is with Netnews, and since newsgroup + descriptions are normally distributed through specially formatted + articles, it is recommended that the internationalisation issues + related to them be addressed in any successor to RFC 1036. + +11. IANA Considerations + + This specification requires IANA to keep a registry of capability + labels. The initial contents of this registry are specified in + Section 3.3.4. As described in Section 3.3.3, labels beginning with + X are reserved for private use, while all other names are expected to + be associated with a specification in an RFC on the standards track + or defining an IESG-approved experimental protocol. + + Different entries in the registry MUST use different capability + labels. + + Different entries in the registry MUST NOT use the same command name. + For this purpose, variants distinguished by a second or subsequent + keyword (e.g., "LIST HEADERS" and "LIST OVERVIEW.FMT") count as + different commands. If there is a need for two extensions to use the + same command, a single harmonised specification MUST be registered. + +12. Security Considerations + + This section is meant to inform application developers, information + providers, and users of the security limitations in NNTP as described + by this document. The discussion does not include definitive + solutions to the problems revealed, though it does make some + suggestions for reducing security risks. + + + + + + + + + +Feather Standards Track [Page 103] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +12.1. Personal and Proprietary Information + + NNTP, because it was created to distribute network news articles, + will forward whatever information is stored in those articles. + Specification of that information is outside this scope of this + document, but it is likely that some personal and/or proprietary + information is available in some of those articles. It is very + important that designers and implementers provide informative + warnings to users so that personal and/or proprietary information in + material that is added automatically to articles (e.g., in headers) + is not disclosed inadvertently. Additionally, effective and easily + understood mechanisms to manage the distribution of news articles + SHOULD be provided to NNTP Server administrators, so that they are + able to report with confidence the likely spread of any particular + set of news articles. + +12.2. Abuse of Server Log Information + + A server is in the position to save session data about a user's + requests that might identify their reading patterns or subjects of + interest. This information is clearly confidential in nature, and + its handling can be constrained by law in certain countries. People + using this protocol to provide data are responsible for ensuring that + such material is not distributed without the permission of any + individuals that are identifiable by the published results. + +12.3. Weak Authentication and Access Control + + There is no user-based or token-based authentication in the basic + NNTP specification. Access is normally controlled by server + configuration files. Those files specify access by using domain + names or IP addresses. However, this specification does permit the + creation of extensions to NNTP for such purposes; one such extension + is [NNTP-AUTH]. While including such mechanisms is optional, doing + so is strongly encouraged. + + Other mechanisms are also available. For example, a proxy server + could be put in place that requires authentication before connecting + via the proxy to the NNTP server. + +12.4. DNS Spoofing + + Many existing NNTP implementations authorize incoming connections by + checking the IP address of that connection against the IP addresses + obtained via DNS lookups of lists of domain names given in local + configuration files. Servers that use this type of authentication + and clients that find a server by doing a DNS lookup of the server + name rely very heavily on the Domain Name Service, and are thus + + + +Feather Standards Track [Page 104] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + generally prone to security attacks based on the deliberate + misassociation of IP addresses and DNS names. Clients and servers + need to be cautious in assuming the continuing validity of an IP + number/DNS name association. + + In particular, NNTP clients and servers SHOULD rely on their name + resolver for confirmation of an IP number/DNS name association, + rather than cache the result of previous host name lookups. Many + platforms already can cache host name lookups locally when + appropriate, and they SHOULD be configured to do so. It is proper + for these lookups to be cached, however, only when the TTL (Time To + Live) information reported by the name server makes it likely that + the cached information will remain useful. + + If NNTP clients or servers cache the results of host name lookups in + order to achieve a performance improvement, they MUST observe the TTL + information reported by DNS. If NNTP clients or servers do not + observe this rule, they could be spoofed when a previously accessed + server's IP address changes. As network renumbering is expected to + become increasingly common, the possibility of this form of attack + will increase. Observing this requirement thus reduces this + potential security vulnerability. + + This requirement also improves the load-balancing behaviour of + clients for replicated servers using the same DNS name and reduces + the likelihood of a user's experiencing failure in accessing sites + that use that strategy. + +12.5. UTF-8 Issues + + UTF-8 [RFC3629] permits only certain sequences of octets and + designates others as either malformed or "illegal". The Unicode + standard identifies a number of security issues related to illegal + sequences and forbids their generation by conforming implementations. + + Implementations of this specification MUST NOT generate malformed or + illegal sequences and SHOULD detect them and take some appropriate + action. This could include the following: + + o Generating a 501 response code. + + o Replacing such sequences by the sequence %xEF.BF.BD, which encodes + the "replacement character" U+FFFD. + + o Closing the connection. + + o Replacing such sequences by a "guessed" valid sequence (based on + properties of the UTF-8 encoding). + + + +Feather Standards Track [Page 105] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + In the last case, the implementation MUST ensure that any replacement + cannot be used to bypass validity or security checks. For example, + the illegal sequence %xC0.A0 is an over-long encoding for space + (%x20). If it is replaced by the correct encoding in a command line, + this needs to happen before the command line is parsed into + individual arguments. If the replacement came after parsing, it + would be possible to generate an argument with an embedded space, + which is forbidden. Use of the "replacement character" does not have + this problem, since it is permitted wherever non-US-ASCII characters + are. Implementations SHOULD use one of the first two solutions where + the general structure of the NNTP stream remains intact and SHOULD + close the connection if it is no longer possible to parse it + sensibly. + +12.6. Caching of Capability Lists + + The CAPABILITIES command provides a capability list, which is + information about the current capabilities of the server. Whenever + there is a relevant change to the server state, the results of this + command are required to change accordingly. + + In most situations, the capabilities list in a given server state + will not change from session to session; for example, a given + extension will be installed permanently on a server. Some clients + may therefore wish to remember which extensions a server supports to + avoid the delay of an additional command and response, particularly + if they open multiple connections in the same session. + + However, information about extensions related to security and privacy + MUST NOT be cached, since this could allow a variety of attacks. + + For example, consider a server that permits the use of cleartext + passwords on links that are encrypted but not otherwise: + + [Initial connection set-up completed.] + [S] 200 NNTP Service Ready, posting permitted + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] NEWNEWS + [S] POST + [S] XENCRYPT + [S] LIST ACTIVE NEWSGROUPS + [S] . + [C] XENCRYPT + [Client and server negotiate encryption on the link] + [S] 283 Encrypted link established + + + +Feather Standards Track [Page 106] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + [C] CAPABILITIES + [S] 101 Capability list: + [S] VERSION 2 + [S] READER + [S] NEWNEWS + [S] POST + [S] XSECRET + [S] LIST ACTIVE NEWSGROUPS + [S] . + [C] XSECRET fred flintstone + [S] 290 Password for fred accepted + + If the client caches the last capabilities list, then on the next + session it will attempt to use XSECRET on an unencrypted link: + + [Initial connection set-up completed.] + [S] 200 NNTP Service Ready, posting permitted + [C] XSECRET fred flintstone + [S] 483 Only permitted on secure links + + This exposes the password to any eavesdropper. While the primary + cause of this is passing a secret without first checking the security + of the link, caching of capability lists can increase the risk. + + Any security extension should include requirements to check the + security state of the link in a manner appropriate to that extension. + + Caching should normally only be considered for anonymous clients that + do not use any security or privacy extensions and for which the time + required for an additional command and response is a noticeable + issue. + +13. Acknowledgements + + This document is the result of much effort by the present and past + members of the NNTP Working Group, chaired by Russ Allbery and Ned + Freed. It could not have been produced without them. + + The author acknowledges the original authors of NNTP as documented in + RFC 977 [RFC977]: Brian Kantor and Phil Lapsey. + + The author gratefully acknowledges the following: + + o The work of the NNTP committee chaired by Eliot Lear. The + organization of this document was influenced by the last available + version from this working group. A special thanks to Eliot for + generously providing the original machine-readable sources for + that document. + + + +Feather Standards Track [Page 107] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + o The work of the DRUMS working group, specifically RFC 1869 + [RFC1869], that drove the original thinking that led to the + CAPABILITIES command and the extensions mechanism detailed in this + document. + + o The authors of RFC 2616 [RFC2616] for providing specific and + relevant examples of security issues that should be considered for + HTTP. Since many of the same considerations exist for NNTP, those + examples that are relevant have been included here with some minor + rewrites. + + o The comments and additional information provided by the following + individuals in preparing one or more of the progenitors of this + document: + + Russ Allbery + Wayne Davison + Chris Lewis + Tom Limoncelli + Eric Schnoebelen + Rich Salz + + This work was motivated by the work of various news reader authors + and news server authors, including those listed below: + + Rick Adams + Original author of the NNTP extensions to the RN news reader and + last maintainer of Bnews. + + Stan Barber + Original author of the NNTP extensions to the news readers that + are part of Bnews. + + Geoff Collyer + Original author of the OVERVIEW database proposal and one of the + original authors of CNEWS. + + Dan Curry + Original author of the xvnews news reader. + + Wayne Davison + Author of the first threading extensions to the RN news reader + (commonly called TRN). + + Geoff Huston + Original author of ANU NEWS. + + + + + +Feather Standards Track [Page 108] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Phil Lapsey + Original author of the UNIX reference implementation for NNTP. + + Iain Lea + Original maintainer of the TIN news reader. + + Chris Lewis + First known implementer of the AUTHINFO GENERIC extension. + + Rich Salz + Original author of INN. + + Henry Spencer + One of the original authors of CNEWS. + + Kim Storm + Original author of the NN news reader. + + Other people who contributed to this document include: + + Matthias Andree + Greg Andruk + Daniel Barclay + Maurizio Codogno + Mark Crispin + Andrew Gierth + Juergen Helbing + Scott Hollenbeck + Urs Janssen + Charles Lindsey + Ade Lovett + David Magda + Ken Murchison + Francois Petillon + Peter Robinson + Rob Siemborski + Howard Swinehart + Ruud van Tol + Jeffrey Vinocur + Erik Warmelink + + The author thanks them all and apologises to anyone omitted. + + Finally, the present author gratefully acknowledges the vast amount + of work put into previous versions by the previous author: + + Stan Barber + + + + +Feather Standards Track [Page 109] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +14. References + +14.1. Normative References + + [ANSI1986] American National Standards Institute, "Coded Character + Set - 7-bit American Standard Code for Information + Interchange", ANSI X3.4, 1986. + + [RFC977] Kantor, B. and P. Lapsley, "Network News Transfer + Protocol", RFC 977, February 1986. + + [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet + Mail Extensions (MIME) Part One: Format of Internet + Message Bodies", RFC 2045, November 1996. + + [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail + Extensions) Part Three: Message Header Extensions for + Non-ASCII Text", RFC 2047, November 1996. + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO + 10646", STD 63, RFC 3629, November 2003. + + [RFC4234] Crocker, D., Ed. and P. Overell, "Augmented BNF for + Syntax Specifications: ABNF", RFC 4234, October 2005. + + [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data + Encodings", RFC 4648, October 2006. + + [TF.686-1] International Telecommunications Union - Radio, + "Glossary, ITU-R Recommendation TF.686-1", + ITU-R Recommendation TF.686-1, October 1997. + +14.2. Informative References + + [NNTP-AUTH] Vinocur, J., Murchison, K., and C. Newman, "Network + News Transfer Protocol (NNTP) Extension for + Authentication", + RFC 4643, October 2006. + + [NNTP-STREAM] Vinocur, J. and K. Murchison, "Network News Transfer + Protocol (NNTP) Extension for Streaming Feeds", + RFC 4644, October 2006. + + + + + + +Feather Standards Track [Page 110] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + [NNTP-TLS] Murchison, K., Vinocur, J., and C. Newman, "Using + Transport Layer Security (TLS) with Network News + Transfer Protocol (NNTP)", RFC 4642, October 2006. + + [RFC1036] Horton, M. and R. Adams, "Standard for interchange of + USENET messages", RFC 1036, December 1987. + + [RFC1305] Mills, D., "Network Time Protocol (Version 3) + Specification, Implementation and Analysis", RFC 1305, + March 1992. + + [RFC1869] Klensin, J., Freed, N., Rose, M., Stefferud, E., and D. + Crocker, "SMTP Service Extensions", STD 10, RFC 1869, + November 1995. + + [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., + Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext + Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. + + [RFC2629] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, + June 1999. + + [RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April + 2001. + + [RFC2980] Barber, S., "Common NNTP Extensions", RFC 2980, October + 2000. + + [ROBE1995] Robertson, R., "FAQ: Overview database / NOV General + Information", January 1995. + + There is no definitive copy of this document known to + the author. It was previously posted as the Usenet + article + + [SALZ1992] Salz, R., "Manual Page for wildmat(3) from the INN 1.4 + distribution, Revision 1.10", April 1992. + + There is no definitive copy of this document known to + the author. + + + + + + + + + + + +Feather Standards Track [Page 111] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +Appendix A. Interaction with Other Specifications + + NNTP is most often used for transferring articles that conform to + RFC 1036 [RFC1036] (such articles are called "Netnews articles" + here). It is also sometimes used for transferring email messages + that conform to RFC 2822 [RFC2822] (such articles are called "email + articles" here). In this situation, articles must conform both to + this specification and to that other one; this appendix describes + some relevant issues. + +A.1. Header Folding + + NNTP allows a header line to be folded (by inserting a CRLF pair) + before any space or TAB character. + + Both email and Netnews articles are required to have at least one + octet other than space or TAB on each header line. Thus, folding can + only happen at one point in each sequence of consecutive spaces or + TABs. Netnews articles are further required to have the header name, + colon, and following space all on the first line; folding may only + happen beyond that space. Finally, some non-conforming software will + remove trailing spaces and TABs from a line. Therefore, it might be + inadvisable to fold a header after a space or TAB. + + For maximum safety, header lines SHOULD conform to the following + syntax rather than to that in Section 9.7. + + + header = header-name ":" SP [header-content] CRLF + header-content = [WS] token *( [CRLF] WS token ) + +A.2. Message-IDs + + Every article handled by an NNTP server MUST have a unique + message-id. For the purposes of this specification, a message-id is + an arbitrary opaque string that merely needs to meet certain + syntactic requirements and is just a way to refer to the article. + + Because there is a significant risk that old articles will be + reinjected into the global Usenet system, RFC 1036 [RFC1036] requires + that message-ids are globally unique for all time. + + This specification states that message-ids are the same if and only + if they consist of the same sequence of octets. Other specifications + may define two different sequences as being equal because they are + putting an interpretation on particular characters. RFC 2822 + [RFC2822] has a concept of "quoted" and "escaped" characters. It + therefore considers the three message-ids: + + + +Feather Standards Track [Page 112] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + + <"ab.cd"@example.com> + <"ab.\cd"@example.com> + + as being identical. Therefore, an NNTP implementation handing email + articles must ensure that only one of these three appears in the + protocol and that the other two are converted to it as and when + necessary, such as when a client checks the results of a NEWNEWS + command against an internal database of message-ids. Note that + RFC 1036 [RFC1036] never treats two different strings as being + identical. Its successor (as of the time of writing) restricts the + syntax of message-ids so that, whenever RFC 2822 would treat two + strings as equivalent, only one of them is valid (in the above + example, only the first string is valid). + + This specification does not describe how the message-id of an article + is determined; it may be deduced from the contents of the article or + derived from some external source. If the server is also conforming + to another specification that contains a definition of message-id + compatible with this one, the server SHOULD use those message-ids. A + common approach, and one that SHOULD be used for email and Netnews + articles, is to extract the message-id from the contents of a header + with name "Message-ID". This may not be as simple as copying the + entire header contents; it may be necessary to strip off comments and + undo quoting, or to reduce "equivalent" message-ids to a canonical + form. + + If an article is obtained through the IHAVE command, there will be a + message-id provided with the command. The server MAY either use it + or determine one from the article contents. However, whichever it + does, it SHOULD ensure that, if the IHAVE command is repeated with + the same argument and article, it will be recognized as a duplicate. + + If an article does not contain a message-id that the server can + identify, it MUST synthesize one. This could, for example, be a + simple sequence number or be based on the date and time when the + article arrived. When email or Netnews articles are handled, a + Message-ID header SHOULD be added to ensure global consistency and + uniqueness. + + Note that, because the message-id might not have been derived from + the Message-ID header in the article, the following example is + legitimate (though unusual): + + + + + + + + +Feather Standards Track [Page 113] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + [C] HEAD <45223423@example.com> + [S] 221 0 <45223423@example.com> + [S] Path: pathost!demo!whitehouse!not-for-mail + [S] Message-ID: <1234@example.net> + [S] From: "Demo User" + [S] Newsgroups: misc.test + [S] Subject: I am just a test article + [S] Date: 6 Oct 1998 04:38:40 -0500 + [S] Organization: An Example Net, Uncertain, Texas + [S] . + +A.3. Article Posting + + As far as NNTP is concerned, the POST and IHAVE commands provide the + same basic facilities in a slightly different way. However, they + have rather different intentions. + + The IHAVE command is intended for transmitting conforming articles + between a system of NNTP servers, with all articles perhaps also + conforming to another specification (e.g., all articles are Netnews + articles). It is expected that the client will already have done any + necessary validation (or that it has in turn obtained the article + from a third party that has done so); therefore, the contents SHOULD + be left unchanged. + + In contrast, the POST command is intended for use when an end-user is + injecting a newly created article into a such a system. The article + being transferred might not be a conforming email or Netnews article, + and the server is expected to validate it and, if necessary, to + convert it to the right form for onward distribution. This is often + done by a separate piece of software on the server installation; if + so, the NNTP server SHOULD pass the incoming article to that software + unaltered, making no attempt to filter characters, to fold or limit + lines, or to process the incoming text otherwise. + + The POST command can fail in various ways, and clients should be + prepared to re-send an article. When doing so, however, it is often + important to ensure (as far as possible) that the same message-id is + allocated to both attempts so that the server, or other servers, can + recognize the two articles as duplicates. In the case of email or + Netnews articles, therefore, the posted article SHOULD contain a + header with the name "Message-ID", and the contents of this header + SHOULD be identical on each attempt. The server SHOULD ensure that + two POSTed articles with the same contents for this header are + recognized as identical and that the same message-id is allocated, + whether or not those contents are suitable for use as the message-id. + + + + + +Feather Standards Track [Page 114] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +Appendix B. Summary of Commands + + This section contains a list of every command defined in this + document, ordered by command name and by indicating capability. + + Ordered by command name: + + +-------------------+-----------------------+---------------+ + | Command | Indicating capability | Definition | + +-------------------+-----------------------+---------------+ + | ARTICLE | READER | Section 6.2.1 | + | BODY | READER | Section 6.2.3 | + | CAPABILITIES | mandatory | Section 5.2 | + | DATE | READER | Section 7.1 | + | GROUP | READER | Section 6.1.1 | + | HDR | HDR | Section 8.5 | + | HEAD | mandatory | Section 6.2.2 | + | HELP | mandatory | Section 7.2 | + | IHAVE | IHAVE | Section 6.3.2 | + | LAST | READER | Section 6.1.3 | + | LIST | LIST | Section 7.6.1 | + | LIST ACTIVE.TIMES | LIST | Section 7.6.4 | + | LIST ACTIVE | LIST | Section 7.6.3 | + | LIST DISTRIB.PATS | LIST | Section 7.6.5 | + | LIST HEADERS | HDR | Section 8.6 | + | LIST NEWSGROUPS | LIST | Section 7.6.6 | + | LIST OVERVIEW.FMT | OVER | Section 8.4 | + | LISTGROUP | READER | Section 6.1.2 | + | MODE READER | MODE-READER | Section 5.3 | + | NEWGROUPS | READER | Section 7.3 | + | NEWNEWS | NEWNEWS | Section 7.4 | + | NEXT | READER | Section 6.1.4 | + | OVER | OVER | Section 8.3 | + | POST | POST | Section 6.3.1 | + | QUIT | mandatory | Section 5.4 | + | STAT | mandatory | Section 6.2.4 | + +-------------------+-----------------------+---------------+ + + + + + + + + + + + + + + +Feather Standards Track [Page 115] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Ordered by indicating capability: + + +-------------------+-----------------------+---------------+ + | Command | Indicating capability | Definition | + +-------------------+-----------------------+---------------+ + | CAPABILITIES | mandatory | Section 5.2 | + | HEAD | mandatory | Section 6.2.2 | + | HELP | mandatory | Section 7.2 | + | QUIT | mandatory | Section 5.4 | + | STAT | mandatory | Section 6.2.4 | + | HDR | HDR | Section 8.5 | + | LIST HEADERS | HDR | Section 8.6 | + | IHAVE | IHAVE | Section 6.3.2 | + | LIST | LIST | Section 7.6.1 | + | LIST ACTIVE | LIST | Section 7.6.3 | + | LIST ACTIVE.TIMES | LIST | Section 7.6.4 | + | LIST DISTRIB.PATS | LIST | Section 7.6.5 | + | LIST NEWSGROUPS | LIST | Section 7.6.6 | + | MODE READER | MODE-READER | Section 5.3 | + | NEWNEWS | NEWNEWS | Section 7.4 | + | OVER | OVER | Section 8.3 | + | LIST OVERVIEW.FMT | OVER | Section 8.4 | + | POST | POST | Section 6.3.1 | + | ARTICLE | READER | Section 6.2.1 | + | BODY | READER | Section 6.2.3 | + | DATE | READER | Section 7.1 | + | GROUP | READER | Section 6.1.1 | + | LAST | READER | Section 6.1.3 | + | LISTGROUP | READER | Section 6.1.2 | + | NEWGROUPS | READER | Section 7.3 | + | NEXT | READER | Section 6.1.4 | + +-------------------+-----------------------+---------------+ + + + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 116] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +Appendix C. Summary of Response Codes + + This section contains a list of every response code defined in this + document and indicates whether it is multi-line, which commands can + generate it, what arguments it has, and what its meaning is. + + Response code 100 (multi-line) + Generated by: HELP + Meaning: help text follows. + + Response code 101 (multi-line) + Generated by: CAPABILITIES + Meaning: capabilities list follows. + + Response code 111 + Generated by: DATE + 1 argument: yyyymmddhhmmss + Meaning: server date and time. + + Response code 200 + Generated by: initial connection, MODE READER + Meaning: service available, posting allowed. + + Response code 201 + Generated by: initial connection, MODE READER + Meaning: service available, posting prohibited. + + Response code 205 + Generated by: QUIT + Meaning: connection closing (the server immediately closes the + connection). + + Response code 211 + The 211 response code has two completely different forms, + depending on which command generated it: + + (not multi-line) + Generated by: GROUP + 4 arguments: number low high group + Meaning: group selected. + + (multi-line) + Generated by: LISTGROUP + 4 arguments: number low high group + Meaning: article numbers follow. + + + + + + +Feather Standards Track [Page 117] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Response code 215 (multi-line) + Generated by: LIST + Meaning: information follows. + + Response code 220 (multi-line) + Generated by: ARTICLE + 2 arguments: n message-id + Meaning: article follows. + + Response code 221 (multi-line) + Generated by: HEAD + 2 arguments: n message-id + Meaning: article headers follow. + + Response code 222 (multi-line) + Generated by: BODY + 2 arguments: n message-id + Meaning: article body follows. + + Response code 223 + Generated by: LAST, NEXT, STAT + 2 arguments: n message-id + Meaning: article exists and selected. + + Response code 224 (multi-line) + Generated by: OVER + Meaning: overview information follows. + + Response code 225 (multi-line) + Generated by: HDR + Meaning: headers follow. + + Response code 230 (multi-line) + Generated by: NEWNEWS + Meaning: list of new articles follows. + + Response code 231 (multi-line) + Generated by: NEWGROUPS + Meaning: list of new newsgroups follows. + + Response code 235 + Generated by: IHAVE (second stage) + Meaning: article transferred OK. + + Response code 240 + Generated by: POST (second stage) + Meaning: article received OK. + + + + +Feather Standards Track [Page 118] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Response code 335 + Generated by: IHAVE (first stage) + Meaning: send article to be transferred. + + Response code 340 + Generated by: POST (first stage) + Meaning: send article to be posted. + + Response code 400 + Generic response and generated by initial connection + Meaning: service not available or no longer available (the server + immediately closes the connection). + + Response code 401 + Generic response + 1 argument: capability-label + Meaning: the server is in the wrong mode; the indicated capability + should be used to change the mode. + + Response code 403 + Generic response + Meaning: internal fault or problem preventing action being taken. + + Response code 411 + Generated by: GROUP, LISTGROUP + Meaning: no such newsgroup. + + Response code 412 + Generated by: ARTICLE, BODY, GROUP, HDR, HEAD, LAST, LISTGROUP, + NEXT, OVER, STAT + Meaning: no newsgroup selected. + + Response code 420 + Generated by: ARTICLE, BODY, HDR, HEAD, LAST, NEXT, OVER, STAT + Meaning: current article number is invalid. + + Response code 421 + Generated by: NEXT + Meaning: no next article in this group. + + Response code 422 + Generated by: LAST + Meaning: no previous article in this group. + + Response code 423 + Generated by: ARTICLE, BODY, HDR, HEAD, OVER, STAT + Meaning: no article with that number or in that range. + + + + +Feather Standards Track [Page 119] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Response code 430 + Generated by: ARTICLE, BODY, HDR, HEAD, OVER, STAT + Meaning: no article with that message-id. + + Response code 435 + Generated by: IHAVE (first stage) + Meaning: article not wanted. + + Response code 436 + Generated by: IHAVE (either stage) + Meaning: transfer not possible (first stage) or failed (second + stage); try again later. + + Response code 437 + Generated by: IHAVE (second stage) + Meaning: transfer rejected; do not retry. + + Response code 440 + Generated by: POST (first stage) + Meaning: posting not permitted. + + Response code 441 + Generated by: POST (second stage) + Meaning: posting failed. + + Response code 480 + Generic response + Meaning: command unavailable until the client has authenticated + itself. + + Response code 483 + Generic response + Meaning: command unavailable until suitable privacy has been + arranged. + + Response code 500 + Generic response + Meaning: unknown command. + + Response code 501 + Generic response + Meaning: syntax error in command. + + + + + + + + + +Feather Standards Track [Page 120] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + Response code 502 + Generic response and generated by initial connection + + Meaning for the initial connection and the MODE READER command: + service permanently unavailable (the server immediately closes the + connection). + + Meaning for all other commands: command not permitted (and there + is no way for the client to change this). + + Response code 503 + Generic response + Meaning: feature not supported. + + Response code 504 + Generic response + Meaning: error in base64-encoding [RFC4648] of an argument. + +Appendix D. Changes from RFC 977 + + In general every attempt has been made to ensure that the protocol + specification in this document is compatible with the version + specified in RFC 977 [RFC977] and the various facilities adopted from + RFC 2980 [RFC2980]. However, there have been a number of changes, + some compatible and some not. + + This appendix lists these changes. It is not guaranteed to be + exhaustive or correct and MUST NOT be relied on. + + o A formal syntax specification (Section 9) has been added. + + o The default character set is changed from US-ASCII [ANSI1986] to + UTF-8 [RFC3629] (note that US-ASCII is a subset of UTF-8). This + matter is discussed further in Section 10. + + o All articles are required to have a message-id, eliminating the + "<0>" placeholder used in RFC 977 in some responses. + + o The newsgroup name matching capabilities already documented in + RFC 977 ("wildmats", Section 4) are clarified and extended. The + new facilities (e.g., the use of commas and exclamation marks) are + allowed wherever wildmats appear in the protocol. + + o Support for pipelining of commands (Section 3.5) is made + mandatory. + + + + + + +Feather Standards Track [Page 121] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + o The principles behind response codes (Section 3.2) have been + tidied up. In particular: + + * the x8x response code family, formerly used for private + extensions, is now reserved for authentication and privacy + extensions; + + * the x9x response code family, formerly intended for debugging + facilities, are now reserved for private extensions; + + * the 502 and 503 generic response codes (Section 3.2.1) have + been redefined; + + * new 401, 403, 480, 483, and 504 generic response codes have + been added. + + o The rules for article numbering (Section 6) have been clarified + (also see Section 6.1.1.2). + + o The SLAVE command (which was ill-defined) is removed from the + protocol. + + o Four-digit years are permitted in the NEWNEWS (Section 7.4) and + NEWGROUPS (Section 7.3) commands (two-digit years are still + permitted). The optional distribution parameter to these commands + has been removed. + + o The LIST command (Section 7.6.1) is greatly extended; the original + is available as LIST ACTIVE, while new variants include + ACTIVE.TIMES, DISTRIB.PATS, and NEWSGROUPS. A new "m" status flag + is added to the LIST ACTIVE response. + + o A new CAPABILITIES command (Section 5.2) allows clients to + determine what facilities are supported by a server. + + o The DATE command (Section 7.1) is adopted from RFC 2980 + effectively unchanged. + + o The LISTGROUP command (Section 6.1.2) is adopted from RFC 2980. + An optional range argument has been added, and the 211 initial + response line now has the same format as the 211 response from the + GROUP command. + + o The MODE READER command (Section 5.3) is adopted from RFC 2980 and + its meaning and effects clarified. + + o The XHDR command in RFC 2980 has been formalised as the new HDR + (Section 8.5) and LIST HEADERS (Section 8.6) commands. + + + +Feather Standards Track [Page 122] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + + o The XOVER command in RFC 2980 has been formalised as the new OVER + (Section 8.3) and LIST OVERVIEW.FMT (Section 8.4) commands. The + former can be applied to a message-id as well as to a range. + + o The concept of article metadata (Section 8.1) has been formalised, + allowing the Bytes and Lines pseudo-headers to be deprecated. + + Client authors should note in particular that lack of support for the + CAPABILITIES command is a good indication that the server does not + support this specification. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 123] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +Author's Address + + Clive D.W. Feather + THUS plc + 322 Regents Park Road + London + N3 2QQ + United Kingdom + + Phone: +44 20 8495 6138 + Fax: +44 870 051 9937 + EMail: clive@demon.net + URI: http://www.davros.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Feather Standards Track [Page 124] + +RFC 3977 Network News Transfer Protocol (NNTP) October 2006 + + +Full Copyright Statement + +Copyright (C) The Internet Society (2006). + + This document is subject to the rights, licenses and restrictions + contained in BCP 78, and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET + ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, + INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE + INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property + + The IETF takes no position regarding the validity or scope of any + Intellectual Property Rights or other rights that might be claimed to + pertain to the implementation or use of the technology described in + this document or the extent to which any license under such rights + might or might not be available; nor does it represent that it has + made any independent effort to identify any such rights. Information + on the procedures with respect to rights in RFC documents can be + found in BCP 78 and BCP 79. + + Copies of IPR disclosures made to the IETF Secretariat and any + assurances of licenses to be made available, or the result of an + attempt made to obtain a general license or permission for the use of + such proprietary rights by implementers or users of this + specification can be obtained from the IETF on-line IPR repository at + http://www.ietf.org/ipr. + + The IETF invites any interested party to bring to its attention any + copyrights, patents or patent applications, or other proprietary + rights that may cover technology that may be required to implement + this standard. Please address the information to the IETF at ietf- + ipr@ietf.org. + +Acknowledgement + + Funding for the RFC Editor function is provided by the IETF + Administrative Support Activity (IASA). + + + + + + + +Feather Standards Track [Page 125] + diff -r f907866f0e4b -r 6fceb66e1ad7 RFC3977.pdf Binary file RFC3977.pdf has changed diff -r f907866f0e4b -r 6fceb66e1ad7 SConstruct --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/SConstruct Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,16 @@ +# This file is used to build and automatically test sonews + +import os +env = Environment() + +env['ENV']['LANG'] = 'en_GB.UTF-8' +env['JAVACFLAGS'] = '-source 1.6 -target 1.6' +env['JAVACLASSPATH'] = '/usr/share/java/junit4.jar:/usr/share/java/glassfish-mail.jar:/usr/share/java/servlet-api-2.5.jar:/usr/share/java/jchart2d.jar:classes' + +# Build Java classes +classes = env.Java(target='classes', source=['org/sonews/']) +test_classes = env.Java(target='classes', source=['test']) + +# Setting dependencies +Depends(test_classes, classes) + diff -r f907866f0e4b -r 6fceb66e1ad7 bin/sonews-web.sh --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/bin/sonews-web.sh Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,10 @@ +#!/bin/bash +SCRIPTROOT=$(dirname $0) +CLASSPATH=$SCRIPTROOT/lib/jchart2d.jar:\ +$SCRIPTROOT/lib/sonews.jar:\ +$SCRIPTROOT/lib/servlet-api-2.5.jar +ARG0=org.sonews.web.SonewsServlet@sonews +ARG1=org.sonews.web.SonewsConfigServlet@sonews/config +ARG2=org.sonews.web.SonewsPeerServlet@sonews/peer +ARG3=org.sonews.web.SonewsChartServlet@sonews/chart +java -cp $CLASSPATH org.sonews.kitten.Main -s $ARG0 -s $ARG1 -s $ARG2 -s $ARG3 diff -r f907866f0e4b -r 6fceb66e1ad7 bin/sonews.sh --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/bin/sonews.sh Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,35 @@ +#!/bin/bash +SCRIPTROOT=$(pwd) +CLASSPATH=$SCRIPTROOT/lib/sonews.jar:\ +$SCRIPTROOT/lib/mysql-connector-java.jar:\ +$SCRIPTROOT/lib/glassfish-mail.jar:\ +$SCRIPTROOT/lib/postgresql.jar + +LOGFILE=sonews.log +PIDFILE=sonews.pid +ARGS=$@ + +MAINCLASS=org.sonews.daemon.Main +JAVA=java + +case "$1" in + start) + echo "Starting sonews Newsserver..." + $JAVA -classpath $CLASSPATH $MAINCLASS $ARGS &> $LOGFILE & + echo $! > $PIDFILE + ;; + stop) + echo "Stopping sonews Newsserver..." + PID=`cat $PIDFILE` + kill -15 $PID + ;; + setup) + $JAVA -classpath $CLASSPATH org.sonews.util.DatabaseSetup + ;; + purge) + $JAVA -classpath $CLASSPATH org.sonews.util.Purger + ;; + + *) + echo "Usage: sonews [start|stop|setup|purge]" +esac diff -r f907866f0e4b -r 6fceb66e1ad7 doc/config.xsl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/config.xsl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,17 @@ + + + + + + + + +

+ Creative Commons License This work by Christian Lins is licensed under a Creative Commons Attribution-Share Alike 3.0 License. +

+ + + + diff -r f907866f0e4b -r 6fceb66e1ad7 doc/makedoc --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/makedoc Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +xmlto --skip-validation html -m config.xsl sonews.xml diff -r f907866f0e4b -r 6fceb66e1ad7 doc/sonews.css --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/sonews.css Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,35 @@ +body +{ + font-family: Liberation Sans,Verdana,sans-serif; + font-size: 11pt; +} + +.screen { + font-family: monospace; + font-size: 1em; + display: block; + padding: 10px; + border: 1px solid #bbb; + background-color: #eee; + color: #000; + overflow: auto; + border-radius: 2.5px; + -moz-border-radius: 2.5px; + margin: 0.5em 2em; + +} + +.programlisting { + font-family: monospace; + font-size: 1em; + display: block; + padding: 10px; + border: 1px solid #bbb; + background-color: #ddd; + color: #000; + overflow: auto; + border-radius: 2.5px; + -moz-border-radius: 2.5px; + margin: 0.5em 2em; +} + diff -r f907866f0e4b -r 6fceb66e1ad7 doc/sonews.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/sonews.xml Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,254 @@ + + + +]> + + sonews Usenet News Server + + sonews is NNTP server than can provide + access to both local and global Usenets newsgroups. It is written in + Java and uses a relational + database as backend. + + + 2009/06/29: + sonews/0.5.0 + (binary tarball, + source tarball) final + released. + The setup is a little clumsy but the software is stable and works well. + + + + Introduction + sonews is a RCF3977 compliant NNTP Usenet server. + It is written in Java and uses a relation database management system + (RDBMS) as backend (currently + PostgreSQL and + MySQL). + sonews is highly multithreaded and uses Java NIO asynchronous sockets + to handle thousands of concurrent connections. + sonews is Free and Open Source Software (FOSS) licensed under the + terms of the + GNU General Public License + Version 3 (or later). + + + History + Ancestor of sonews is probably the Neat NNTP Daemon (n3tpd) + although there is very little code in sonews that can be identified + as direct derivation. + sonews was developed as diploma thesis project of Christian Lins at + StarOffice development + in Hamburg and is now a Free Software project. + + + + Roadmap + + sonews/0.6 + Planned to implement the XPAT command for searching, correctly + hashed Message-Ids and a news purging command. + See Bugtracker for + issues with target sonews/0.6.x. + + + + + + Installation and initial setup + + Download & Installation + + Debian based systems + You can install sonews with + APT + easily. + Add the following line to /etc/apt/sources.list: + deb http://packages.xerxys.info/debian/ unstable main + + And add the GPG-Key for package authentification, see + Xerxys Debian Repository + for more details. + Then force an update of your local package list: + # apt-get update + + To install sonews and all prerequisites issue the following command: + # apt-get install sonews + + This method should work for all recent Debian-based distributions +(Debian, Ubuntu, etc.). + + + + Other *nix systems + See Files Section for recent binary and source tarballs. + Use the binary archive and extract it in a directory of your choice. Make sure your system +provides the necessary prerequisites: + + + Java6 compatible runtime (JRE) + + + Java Mail API implementation, e.g. Sun Java Mail. +GNU JavaMail has a broken POP3 Provider and does not work with sonews. + + + JSP Servlet Container (e.g. + Kitten) [optional] + + + + + + + Initial database setup + Before you start sonews, you must prepare the database. Currently sonews is known +to work with PostgreSQL and MySQL. + It is highly recommended to create an own database for every sonews instance, e.g. +called 'sonews'. Additionally, it is recommended to create a unique database user +for sonews, e.g. 'sonewsuser'. Please do not use the root user for sonews! +The sonews user needs rights for SELECT, INSERT and UPDATE statements. +Refer to the database's manual for instructions. + You will find the SQL Schema definitions in the helpers subdirectory of +the source and binary distributions. You can create the tables manually using +this templates or you can use the setup helper: + user@debian$ sonews setup + + or on other *nix systems: + user@nix$ java -jar sonews.jar org.sonews.util.DatabaseSetup + + The tool will ask for some information about your database environment, +connect to the database, create the tables and creates a default bootstrap +config file called sonews.conf. + + + + + Running sonews + + Configuration + There is a bootstrap configuration in /etc/sonews/sonews.conf and a regular configuration +in the database table config. + There are various configuration values that can be adapted: + + + ‘sonews.article.maxsize’ + + Maximum allowed body size of a news message given in kilobytes. Please note that +for MySQL the ‘max_allowed_packet’ configuration variable must +be set to a value higher than ‘sonews.article.maxsize’ otherwise posting +of large mails will fail. + + + + ‘sonews.debug’ + + + If set to true every(!) data going through sonews' socket + is written to sonews.log. After a night the logfile can be + several gigabytes large, so be careful with this setting. + + + + + ‘sonews.hostname’ + + Canonical name of the server instance. This variable is part of the server's +hello message to the client and used to generate Message-Ids. + + + + ‘sonews.timeout’ + + Socket timeout for client connections in seconds. + + + + ‘sonews.port’ + + Listening port of sonews daemon. + + + + + + + Command line arguments + If you like to start sonews directly, you can use one of the following +arguments: + java -jar sonews.jar [arguments] + where arguments: + -c|-config <path to config file> if custom config file preferred + -dumpjdbcdriver Prints out a list of available JDBC drivers + -feed Enables feed daemon for pulling news from peer servers + -h|-help This output + -mlgw Enables the Mailinglist Gateway poller + -p portnumber Port on which sonews is listening for incoming connections. + Overrides port settings in config file and database. + + + + + + Webinterface + The package sonews-web provides an optional webinterface that can be used to +review statistical information and configuration values of sonews. + sonews-web start|stop + + The webinterface uses the the lightweight Servlet Container Kitten and is +per default listening on HTTP-Port 8080 (go to http://localhost:8080/sonews). + + + + + Development + You're welcome to create patches with bugfixes or additional features. The +Mercurial DSCM makes this step an easy task. + Just clone the public Mercurial repository: + hg clone http://code.xerxys.info:8000/hg/sonews/trunk sonews-trunk + + Then make your changes, create a bundle of changesets and send this to me via email. +Or ask for push access to the public repository. + + There is a nightly generated Javadoc API documentation that will help + you to get in touch with the sonews source. + + Some debugging hints: if the server blocks and does not longer respond you +probably found a deadlock. Do not kill the process with "kill -9 <pid>" +but send a SIGQUIT signal with "kill -3 <pid>" and the Java VM will output +a stracktrace of all threads. This output is the most valuable information to +fix the deadlock. + + + Contributors + Maintainer and project lead: +Christian Lins (contact christian.lins (at) fh-osnabrueck.de) + + + + Sponsors + The author thanks Sun Microsystems for fully +financing the first version of sonews. A really free software supporting company! + If you like to support sonews with a donation of any kind (hardware, books, money, donuts,...), + feel free to contact the project leader. +A friendly email or a bug report is most welcome, too :-) + + + + + Links and further information + + + Bugtracker, register necessary, see project 'sonews'. + + + Sun Microsystems, friendly sponsor. + + + University of Applied Sciences Osnabrueck + + + + diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/copyright --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/copyright Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,23 @@ +Upstream Author: + + Christian Lins + +Copyright: + + Copyright (C) 2009 Christian Lins + +License: + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/database_mysql5_tmpl.sql --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/database_mysql5_tmpl.sql Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,158 @@ +/* + Create a database at first: + CREATE DATABASE sonews CHARACTER SET utf8 +*/ + +/* + flags: + If bit 0 is set, groups is a mirrorred mailing list. + If not set default newsgroup. + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE groups +( + group_id SERIAL, + name VARCHAR(80) NOT NULL, + flags TINYINT UNSIGNED DEFAULT 0, + + PRIMARY KEY(group_id), + UNIQUE(name) +) +ENGINE = INNODB +CHARACTER SET utf8; + +CREATE TABLE articles +( + article_id INT, + body LONGBLOB, + + PRIMARY KEY(article_id) +) +ENGINE = INNODB +CHARACTER SET utf8; + +CREATE TABLE article_ids +( + article_id INT REFERENCES articles.article_id ON DELETE CASCADE, + message_id VARCHAR(255), + + PRIMARY KEY(article_id), + UNIQUE(message_id) +) +ENGINE = INNODB +CHARACTER SET utf8; + +CREATE TABLE headers +( + article_id INT REFERENCES articles.article_id ON DELETE CASCADE, + header_key VARCHAR(255), + header_value TEXT, /* Max. 64k */ + header_index INT, + + PRIMARY KEY(article_id, header_key, header_index) +) +ENGINE = INNODB +CHARACTER SET utf8; + +/* + Normalization: 1NF, 2NF +*/ +CREATE TABLE postings +( + group_id INTEGER, + article_id INTEGER REFERENCES articles.article_id ON DELETE CASCADE, + article_index INTEGER NOT NULL, + + PRIMARY KEY(group_id, article_id) +) +ENGINE = INNODB +CHARACTER SET utf8; + +/* + Table for association of newsgroups and mailing-lists + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE groups2list +( + group_id INTEGER REFERENCES groups.group_id ON DELETE CASCADE, + listaddress VARCHAR(255), + + PRIMARY KEY(group_id, listaddress), + UNIQUE(listaddress) +) +ENGINE = INNODB +CHARACTER SET utf8; + +/* + Configuration table, containing key/value pairs + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE config +( + config_key VARCHAR(255), + config_value TEXT, + + PRIMARY KEY(config_key) +) +ENGINE = INNODB +CHARACTER SET utf8; + +/* + Newsserver peers + feedtype: 0: pullfeed 1: pushfeed + Normalization: 1NF (atomic values), 2NF +*/ +CREATE TABLE peers +( + peer_id SERIAL, + host VARCHAR(255), + port SMALLINT UNSIGNED, + + PRIMARY KEY(peer_id), + UNIQUE(host, port) +) +ENGINE = INNODB +CHARACTER SET utf8; + +/* + List of newsgroups to feed into sonews + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE peer_subscriptions +( + peer_id INTEGER REFERENCES peers.peer_id ON DELETE CASCADE, + group_id INTEGER REFERENCES groups.group_id ON DELETE CASCADE, + feedtype TINYINT UNSIGNED DEFAULT 0, + + PRIMARY KEY(peer_id, group_id, feedtype) +) +ENGINE = INNODB +CHARACTER SET utf8; + +/* + Tables for server event statistics + + Possible statistic keys: + 1=CONNECTIONS (active connections) + 2=POSTED_NEWS (directly to the server posted unique messages) + 3=GATEWAYED_NEWS (posted unique message gateways through the ML-gateway) + 4=FEEDED_NEWS (unique messages feed via NNTP) + + The server will create snapshots of the above data. + + Normalization: 1NF, 2NF +*/ +CREATE TABLE events +( + event_time BIGINT UNSIGNED, /* time of this snapshot */ + event_key TINYINT UNSIGNED, /* which data */ + group_id INT REFERENCES groups.group_id ON DELETE CASCADE, + + PRIMARY KEY(event_time, event_key) +) +ENGINE = INNODB +CHARACTER SET utf8; diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/database_postgresql8_tmpl.sql --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/database_postgresql8_tmpl.sql Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,138 @@ +/* + Create a database at first: + CREATE DATABASE sonews ENCODING 'UTF8'; +*/ + +/* + flags: + If bit 0 is set, groups is a mirrorred mailing list. + If not set default newsgroup. + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE groups +( + group_id SERIAL, + name VARCHAR(80) NOT NULL, + flags SMALLINT DEFAULT 0, + + PRIMARY KEY(group_id), + UNIQUE(name) +); + +CREATE TABLE articles +( + article_id INT, + body BYTEA, + + PRIMARY KEY(article_id) +); + +CREATE TABLE article_ids +( + article_id INT REFERENCES articles(article_id) ON DELETE CASCADE, + message_id VARCHAR(255), + + PRIMARY KEY(article_id), + UNIQUE(message_id) +); + +CREATE TABLE headers +( + article_id INT REFERENCES articles(article_id) ON DELETE CASCADE, + header_key VARCHAR(255), + header_value TEXT, + header_index INT, + + PRIMARY KEY(article_id, header_key, header_index) +); + +/* + Normalization: 1NF, 2NF +*/ +CREATE TABLE postings +( + group_id INTEGER, + article_id INTEGER REFERENCES articles (article_id) ON DELETE CASCADE, + article_index INTEGER NOT NULL, + + PRIMARY KEY(group_id, article_id) +); + +/* + Table for association of newsgroups and mailing-lists + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE groups2list +( + group_id INTEGER REFERENCES groups(group_id) ON DELETE CASCADE, + listaddress VARCHAR(255), + + PRIMARY KEY(group_id, listaddress), + UNIQUE(listaddress) +); + +/* + Configuration table, containing key/value pairs + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE config +( + config_key VARCHAR(255), + config_value TEXT, + + PRIMARY KEY(config_key) +); + +/* + Newsserver peers + + Normalization: 1NF (atomic values), 2NF +*/ +CREATE TABLE peers +( + peer_id SERIAL, + host VARCHAR(255), + port SMALLINT, + + PRIMARY KEY(peer_id), + UNIQUE(host, port) +); + +/* + List of newsgroups to feed into sonews + + Normalization: 1NF, 2NF, 3NF +*/ +CREATE TABLE peer_subscriptions +( + peer_id INTEGER REFERENCES peers (peer_id) ON DELETE CASCADE, + group_id INTEGER REFERENCES groups (group_id) ON DELETE CASCADE, + feedtype SMALLINT DEFAULT 0, /* 0: pullfeed; 1: pushfeed */ + + PRIMARY KEY(peer_id, group_id, feedtype) +); + +/* + Tables for server event statistics + + Possible statistic keys: + 1=CONNECTIONS (active connections) + 2=POSTED_NEWS (directly to the server posted unique messages) + 3=GATEWAYED_NEWS (posted unique message gateways through the ML-gateway) + 4=FEEDED_NEWS (unique messages feed via NNTP) + + The server will create snapshots of the above data. + + Normalization: 1NF, 2NF +*/ +CREATE TABLE events +( + event_time BIGINT, /* time of this snapshot */ + event_key SMALLINT, /* which data */ + group_id INT REFERENCES groups(group_id) ON DELETE CASCADE, + + PRIMARY KEY(event_time, event_key) +); diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/helptext --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/helptext Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,12 @@ +Welcome to sonews help system + +Here is a short overview of supported NNTP commands of this newsserver: + +ARTICLE + Retrieve article including its head + +GROUP + Change currently selected group + +POST + Post an article to a newsgroup \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/sonews --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/sonews Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,42 @@ +#!/bin/bash + +CLASSPATH=/usr/share/java/sonews.jar:\ +/usr/share/java/mysql-connector-java.jar:\ +/usr/share/java/glassfish-mail.jar:\ +/usr/share/java/postgresql.jar + +LOGFILE=/var/log/sonews.log +PIDFILE=/var/run/sonews.pid +ARGS="-mlgw -c /etc/sonews/sonews.conf -feed" + +MAINCLASS=org.sonews.daemon.Main +JAVA=java + +case "$1" in + start) + echo "Starting sonews Newsserver..." + $JAVA -classpath $CLASSPATH $MAINCLASS $ARGS &> $LOGFILE & + PID=$! + echo $PID > $PIDFILE + ;; + stop) + echo "Stopping sonews Newsserver..." + PID=`cat $PIDFILE` + STOPRES=0 + while [ $STOPRES -le 0 ] + do + kill -15 $PID &> /dev/null + STOPRES=$? + sleep 1 + done + echo "done." + ;; + setup) + $JAVA -classpath $CLASSPATH org.sonews.util.DatabaseSetup + ;; + purge) + $JAVA -classpath $CLASSPATH org.sonews.util.Purger + ;; + *) + echo "Usage: sonews [start|stop|restart|setup|purge]" +esac diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/sonews-web --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/sonews-web Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,7 @@ +#!/bin/bash +export CLASSPATH=/usr/share/java/jchart2d.jar:/usr/share/java/sonews.jar +ARG0=org.sonews.web.SonewsServlet@sonews +ARG1=org.sonews.web.SonewsConfigServlet@sonews/config +ARG2=org.sonews.web.SonewsPeerServlet@sonews/peer +ARG3=org.sonews.web.SonewsChartServlet@sonews/chart +/usr/bin/kitten -s $ARG0 -s $ARG1 -s $ARG2 -s $ARG3 diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/sonews.conf.sample --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/sonews.conf.sample Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,4 @@ +sonews.storage.database=jdbc:mysql://localhost/sonews +sonews.storage.user=sonews +sonews.storage.dbmsdriver=com.mysql.jdbc.Driver +sonews.storage.password=mySecret \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 helpers/usage --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/usage Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,8 @@ +java -jar sonews.jar [arguments] + where arguments: + -c|-config if custom config file preferred + -dumpjdbcdriver Prints out a list of available JDBC drivers + -feed Enables feed daemon for pulling news from peer servers + -h|-help This output + -mlgw Enables the Mailinglist Gateway poller + -useaux Enables an additional secondary port for listening diff -r f907866f0e4b -r 6fceb66e1ad7 makedeb --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/makedeb Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,45 @@ +#!/bin/bash -x +PACKAGE_ROOT=sonews + +# Compile classes +scons + +# Create JAR files; this cannot be done with SCons, +# because Scons looses inner classes. +jar -cf sonews.jar -C classes/ org/ +jar -ufe sonews.jar org.sonews.daemon.Main +jar -cf test.jar -C classes/ test/ +jar -ufe test.jar test.TestBench +jar -cf sonews-helpers.jar helpers/ +jar -uf sonews.jar org/sonews/web/*.tmpl + +# Create faked root for packaging +sudo rm -r $PACKAGE_ROOT/ +mkdir -p $PACKAGE_ROOT/usr/share/java +mkdir -p $PACKAGE_ROOT/usr/bin +mkdir -p $PACKAGE_ROOT/etc/sonews +mkdir -p $PACKAGE_ROOT/usr/share/doc/sonews/ +cp -r DEBIAN $PACKAGE_ROOT/ +cp helpers/sonews $PACKAGE_ROOT/usr/bin/sonews +cp helpers/sonews.conf.sample $PACKAGE_ROOT/etc/sonews/sonews.conf +cp helpers/copyright $PACKAGE_ROOT/usr/share/doc/sonews/ +cp sonews*.jar $PACKAGE_ROOT/usr/share/java/ + +sudo chown root:root -R $PACKAGE_ROOT/ + +dpkg-deb --build $PACKAGE_ROOT + +# Cleanup +sudo rm -r $PACKAGE_ROOT +rm -r classes/ + +# Create metapackage sonews-web +PACKAGE_ROOT=sonews-web +mkdir $PACKAGE_ROOT +cp -r DEBIAN-web $PACKAGE_ROOT/DEBIAN +dpkg-deb --build $PACKAGE_ROOT +rm -r $PACKAGE_ROOT + +# Check debs +lintian sonews.deb +lintian sonews-web.deb diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/AbstractDaemon.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/AbstractDaemon.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,105 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.sql.SQLException; +import org.sonews.daemon.storage.Database; +import org.sonews.util.Log; + +/** + * Base class of all sonews threads. + * Instances of this class will be automatically registered at the ShutdownHook + * to be cleanly exited when the server is forced to exit. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public abstract class AbstractDaemon extends Thread +{ + + /** This variable is write synchronized through setRunning */ + private boolean isRunning = false; + + /** + * Protected constructor. Will be called by derived classes. + */ + protected AbstractDaemon() + { + setDaemon(true); // VM will exit when all threads are daemons + setName(getClass().getSimpleName()); + } + + /** + * @return true if shutdown() was not yet called. + */ + public boolean isRunning() + { + synchronized(this) + { + return this.isRunning; + } + } + + /** + * Marks this thread to exit soon. Closes the associated Database connection + * if available. + * @throws java.sql.SQLException + */ + void shutdownNow() + throws SQLException + { + synchronized(this) + { + this.isRunning = false; + Database db = Database.getInstance(false); + if(db != null) + { + db.shutdown(); + } + } + } + + /** + * Calls shutdownNow() but catches SQLExceptions if occurring. + */ + public void shutdown() + { + try + { + shutdownNow(); + } + catch(SQLException ex) + { + Log.msg(ex, true); + } + } + + /** + * Starts this daemon. + */ + @Override + public void start() + { + synchronized(this) + { + this.isRunning = true; + } + super.start(); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/BootstrapConfig.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/BootstrapConfig.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,194 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.FileOutputStream; +import java.io.IOException; +import java.util.Properties; +import org.sonews.util.AbstractConfig; + +/** + * Manages the bootstrap configuration. It MUST contain all config values + * that are needed to establish a database connection. + * For further configuration values use the Config class instead as that class + * stores its values within the database. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class BootstrapConfig extends AbstractConfig +{ + + /** Key constant. If value is "true" every I/O is written to logfile + * (which is a lot!) + */ + public static final String DEBUG = "sonews.debug"; + + /** Key constant. Value is classname of the JDBC driver */ + public static final String STORAGE_DBMSDRIVER = "sonews.storage.dbmsdriver"; + + /** Key constant. Value is JDBC connect String to the database. */ + public static final String STORAGE_DATABASE = "sonews.storage.database"; + + /** Key constant. Value is the username for the DBMS. */ + public static final String STORAGE_USER = "sonews.storage.user"; + + /** Key constant. Value is the password for the DBMS. */ + public static final String STORAGE_PASSWORD = "sonews.storage.password"; + + /** Key constant. Value is the name of the host which is allowed to use the + * XDAEMON command; default: "localhost" */ + public static final String XDAEMON_HOST = "sonews.xdaemon.host"; + + /** The config key for the filename of the logfile */ + public static final String LOGFILE = "sonews.log"; + + /** The filename of the config file that is loaded on startup */ + public static volatile String FILE = "sonews.conf"; + + private static final Properties defaultConfig = new Properties(); + + private static BootstrapConfig instance = null; + + static + { + // Set some default values + defaultConfig.setProperty(STORAGE_DATABASE, "jdbc:mysql://localhost/sonews"); + defaultConfig.setProperty(STORAGE_DBMSDRIVER, "com.mysql.jdbc.Driver"); + defaultConfig.setProperty(STORAGE_USER, "sonews_user"); + defaultConfig.setProperty(STORAGE_PASSWORD, "mysecret"); + defaultConfig.setProperty(DEBUG, "false"); + } + + /** + * Note: this method is not thread-safe + * @return A Config instance + */ + public static synchronized BootstrapConfig getInstance() + { + if(instance == null) + { + instance = new BootstrapConfig(); + } + return instance; + } + + // Every config instance is initialized with the default values. + private final Properties settings = (Properties)defaultConfig.clone(); + + /** + * Config is a singelton class with only one instance at time. + * So the constructor is private to prevent the creation of more + * then one Config instance. + * @see Config.getInstance() to retrieve an instance of Config + */ + private BootstrapConfig() + { + try + { + // Load settings from file + load(); + } + catch(IOException ex) + { + ex.printStackTrace(); + } + } + + /** + * Loads the configuration from the config file. By default this is done + * by the (private) constructor but it can be useful to reload the config + * by invoking this method. + * @throws IOException + */ + public void load() + throws IOException + { + FileInputStream in = null; + + try + { + in = new FileInputStream(FILE); + settings.load(in); + } + catch (FileNotFoundException e) + { + // MUST NOT use Log otherwise endless loop + System.err.println(e.getMessage()); + save(); + } + finally + { + if(in != null) + in.close(); + } + } + + /** + * Saves this Config to the config file. By default this is done + * at program end. + * @throws FileNotFoundException + * @throws IOException + */ + public void save() throws FileNotFoundException, IOException + { + FileOutputStream out = null; + try + { + out = new FileOutputStream(FILE); + settings.store(out, "SONEWS Config File"); + out.flush(); + } + catch(IOException ex) + { + throw ex; + } + finally + { + if(out != null) + out.close(); + } + } + + /** + * Returns the value that is stored within this config + * identified by the given key. If the key cannot be found + * the default value is returned. + * @param key Key to identify the value. + * @param def The default value that is returned if the key + * is not found in this Config. + * @return + */ + public String get(String key, String def) + { + return settings.getProperty(key, def); + } + + /** + * Sets the value for a given key. + * @param key + * @param value + */ + public void set(final String key, final String value) + { + settings.setProperty(key, value); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/ChannelLineBuffers.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/ChannelLineBuffers.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,270 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.nio.ByteBuffer; +import java.nio.channels.ClosedChannelException; +import java.util.ArrayList; +import java.util.List; + +/** + * Class holding ByteBuffers for SocketChannels/NNTPConnection. + * Due to the complex nature of AIO/NIO we must properly handle the line + * buffers for the input and output of the SocketChannels. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ChannelLineBuffers +{ + + /** + * Size of one small buffer; + * per default this is 512 bytes to fit one standard line. + */ + public static final int BUFFER_SIZE = 512; + + private static int maxCachedBuffers = 2048; // Cached buffers maximum + + private static final List freeSmallBuffers + = new ArrayList(maxCachedBuffers); + + /** + * Allocates a predefined number of direct ByteBuffers (allocated via + * ByteBuffer.allocateDirect()). This method is Thread-safe, but should only + * called at startup. + */ + public static void allocateDirect() + { + synchronized(freeSmallBuffers) + { + for(int n = 0; n < maxCachedBuffers; n++) + { + ByteBuffer buffer = ByteBuffer.allocateDirect(BUFFER_SIZE); + freeSmallBuffers.add(buffer); + } + } + } + + private ByteBuffer inputBuffer = newLineBuffer(); + private List outputBuffers = new ArrayList(); + + /** + * Add the given ByteBuffer to the list of buffers to be send to the client. + * This method is Thread-safe. + * @param buffer + * @throws java.nio.channels.ClosedChannelException If the client channel was + * already closed. + */ + public void addOutputBuffer(ByteBuffer buffer) + throws ClosedChannelException + { + if(outputBuffers == null) + { + throw new ClosedChannelException(); + } + + synchronized(outputBuffers) + { + outputBuffers.add(buffer); + } + } + + /** + * Currently a channel has only one input buffer. This *may* be a bottleneck + * and should investigated in the future. + * @param channel + * @return The input buffer associated with given channel. + */ + public ByteBuffer getInputBuffer() + { + return inputBuffer; + } + + /** + * Returns the current output buffer for writing(!) to SocketChannel. + * @param channel + * @return The next input buffer that contains unprocessed data or null + * if the connection was closed or there are no more unprocessed buffers. + */ + public ByteBuffer getOutputBuffer() + { + synchronized(outputBuffers) + { + if(outputBuffers == null || outputBuffers.isEmpty()) + { + return null; + } + else + { + ByteBuffer buffer = outputBuffers.get(0); + if(buffer.remaining() == 0) + { + outputBuffers.remove(0); + // Add old buffers to the list of free buffers + recycleBuffer(buffer); + buffer = getOutputBuffer(); + } + return buffer; + } + } + } + + /** + * Goes through the input buffer of the given channel and searches + * for next line terminator. If a '\n' is found, the bytes up to the + * line terminator are returned as array of bytes (the line terminator + * is omitted). If none is found the method returns null. + * @param channel + * @return A ByteBuffer wrapping the line. + */ + ByteBuffer nextInputLine() + { + if(inputBuffer == null) + { + return null; + } + + synchronized(inputBuffer) + { + ByteBuffer buffer = inputBuffer; + + // Mark the current write position + int mark = buffer.position(); + + // Set position to 0 and limit to current position + buffer.flip(); + + ByteBuffer lineBuffer = newLineBuffer(); + + while (buffer.position() < buffer.limit()) + { + byte b = buffer.get(); + if (b == 10) // '\n' + { + // The bytes between the buffer's current position and its limit, + // if any, are copied to the beginning of the buffer. That is, the + // byte at index p = position() is copied to index zero, the byte at + // index p + 1 is copied to index one, and so forth until the byte + // at index limit() - 1 is copied to index n = limit() - 1 - p. + // The buffer's position is then set to n+1 and its limit is set to + // its capacity. + buffer.compact(); + + lineBuffer.flip(); // limit to position, position to 0 + return lineBuffer; + } + else + { + lineBuffer.put(b); + } + } + + buffer.limit(BUFFER_SIZE); + buffer.position(mark); + + if(buffer.hasRemaining()) + { + return null; + } + else + { + // In the first 512 was no newline found, so the input is not standard + // compliant. We return the current buffer as new line and add a space + // to the beginning of the next line which corrects some overlong header + // lines. + inputBuffer = newLineBuffer(); + inputBuffer.put((byte)' '); + buffer.flip(); + return buffer; + } + } + } + + /** + * Returns a at least 512 bytes long ByteBuffer ready for usage. + * The method first try to reuse an already allocated (cached) buffer but + * if that fails returns a newly allocated direct buffer. + * Use recycleBuffer() method when you do not longer use the allocated buffer. + */ + static ByteBuffer newLineBuffer() + { + ByteBuffer buf = null; + synchronized(freeSmallBuffers) + { + if(!freeSmallBuffers.isEmpty()) + { + buf = freeSmallBuffers.remove(0); + } + } + + if(buf == null) + { + // Allocate a non-direct buffer + buf = ByteBuffer.allocate(BUFFER_SIZE); + } + + assert buf.position() == 0; + assert buf.limit() >= BUFFER_SIZE; + + return buf; + } + + /** + * Adds the given buffer to the list of free buffers if it is a valuable + * direct allocated buffer. + * @param buffer + */ + public static void recycleBuffer(ByteBuffer buffer) + { + assert buffer != null; + assert buffer.capacity() >= BUFFER_SIZE; + + if(buffer.isDirect()) + { + // Add old buffers to the list of free buffers + synchronized(freeSmallBuffers) + { + buffer.clear(); // Set position to 0 and limit to capacity + freeSmallBuffers.add(buffer); + } + } // if(buffer.isDirect()) + } + + /** + * Recycles all buffers of this ChannelLineBuffers object. + */ + public void recycleBuffers() + { + synchronized(inputBuffer) + { + recycleBuffer(inputBuffer); + this.inputBuffer = null; + } + + synchronized(outputBuffers) + { + for(ByteBuffer buf : outputBuffers) + { + recycleBuffer(buf); + } + outputBuffers = null; + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/ChannelReader.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/ChannelReader.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,202 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.channels.CancelledKeyException; +import java.nio.channels.SelectionKey; +import java.nio.channels.Selector; +import java.nio.channels.SocketChannel; +import java.util.Iterator; +import java.util.Set; + +/** + * A Thread task listening for OP_READ events from SocketChannels. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class ChannelReader extends AbstractDaemon +{ + + private static ChannelReader instance = new ChannelReader(); + + /** + * @return Active ChannelReader instance. + */ + public static ChannelReader getInstance() + { + return instance; + } + + private Selector selector = null; + + protected ChannelReader() + { + } + + /** + * Sets the selector which is used by this reader to determine the channel + * to read from. + * @param selector + */ + public void setSelector(final Selector selector) + { + this.selector = selector; + } + + /** + * Run loop. Blocks until some data is available in a channel. + */ + @Override + public void run() + { + assert selector != null; + + while(isRunning()) + { + try + { + // select() blocks until some SelectableChannels are ready for + // processing. There is no need to lock the selector as we have only + // one thread per selector. + selector.select(); + + // Get list of selection keys with pending events. + // Note: the selected key set is not thread-safe + SocketChannel channel = null; + NNTPConnection conn = null; + final Set selKeys = selector.selectedKeys(); + SelectionKey selKey = null; + + synchronized (selKeys) + { + Iterator it = selKeys.iterator(); + + // Process the first pending event + while (it.hasNext()) + { + selKey = (SelectionKey) it.next(); + channel = (SocketChannel) selKey.channel(); + conn = Connections.getInstance().get(channel); + + // Because we cannot lock the selKey as that would cause a deadlock + // we lock the connection. To preserve the order of the received + // byte blocks a selection key for a connection that has pending + // read events is skipped. + if (conn == null || conn.tryReadLock()) + { + // Remove from set to indicate that it's being processed + it.remove(); + if (conn != null) + { + break; // End while loop + } + } + else + { + selKey = null; + channel = null; + conn = null; + } + } + } + + // Do not lock the selKeys while processing because this causes + // a deadlock in sun.nio.ch.SelectorImpl.lockAndDoSelect() + if (selKey != null && channel != null && conn != null) + { + processSelectionKey(conn, channel, selKey); + conn.unlockReadLock(); + } + + } + catch(CancelledKeyException ex) + { + Log.msg("ChannelReader.run(): " + ex, false); + if(Log.isDebug()) + { + ex.printStackTrace(); + } + } + catch(Exception ex) + { + ex.printStackTrace(); + } + + // Eventually wait for a register operation + synchronized (NNTPDaemon.RegisterGate) + { + // Do nothing; FindBugs may warn about an empty synchronized + // statement, but we cannot use a wait()/notify() mechanism here. + // If we used something like RegisterGate.wait() we block here + // until the NNTPDaemon calls notify(). But the daemon only + // calls notify() if itself is NOT blocked in the listening socket. + } + } // while(isRunning()) + } + + private void processSelectionKey(final NNTPConnection connection, + final SocketChannel socketChannel, final SelectionKey selKey) + throws InterruptedException, IOException + { + assert selKey != null; + assert selKey.isReadable(); + + // Some bytes are available for reading + if(selKey.isValid()) + { + // Lock the channel + //synchronized(socketChannel) + { + // Read the data into the appropriate buffer + ByteBuffer buf = connection.getInputBuffer(); + int read = -1; + try + { + read = socketChannel.read(buf); + } + catch(Exception ex) + { + Log.msg("ChannelReader.processSelectionKey(): " + ex, false); + if(Log.isDebug()) + { + ex.printStackTrace(); + } + } + + if(read == -1) // End of stream + { + selKey.cancel(); + } + else if(read > 0) // If some data was read + { + ConnectionWorker.addChannel(socketChannel); + } + } + } + else + { + // Should not happen + Log.msg(selKey, false); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/ChannelWriter.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/ChannelWriter.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,207 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.channels.CancelledKeyException; +import java.nio.channels.SelectionKey; +import java.nio.channels.Selector; +import java.nio.channels.SocketChannel; +import java.util.Iterator; + +/** + * A Thread task that processes OP_WRITE events for SocketChannels. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class ChannelWriter extends AbstractDaemon +{ + + private static ChannelWriter instance = new ChannelWriter(); + + /** + * @return Returns the active ChannelWriter instance. + */ + public static ChannelWriter getInstance() + { + return instance; + } + + private Selector selector = null; + + protected ChannelWriter() + { + } + + /** + * @return Selector associated with this instance. + */ + public Selector getSelector() + { + return this.selector; + } + + /** + * Sets the selector that is used by this ChannelWriter. + * @param selector + */ + public void setSelector(final Selector selector) + { + this.selector = selector; + } + + /** + * Run loop. + */ + @Override + public void run() + { + assert selector != null; + + while(isRunning()) + { + try + { + SelectionKey selKey = null; + SocketChannel socketChannel = null; + NNTPConnection connection = null; + + // select() blocks until some SelectableChannels are ready for + // processing. There is no need to synchronize the selector as we + // have only one thread per selector. + selector.select(); // The return value of select can be ignored + + // Get list of selection keys with pending OP_WRITE events. + // The keySET is not thread-safe whereas the keys itself are. + Iterator it = selector.selectedKeys().iterator(); + + while (it.hasNext()) + { + // We remove the first event from the set and store it for + // later processing. + selKey = (SelectionKey) it.next(); + socketChannel = (SocketChannel) selKey.channel(); + connection = Connections.getInstance().get(socketChannel); + + it.remove(); + if (connection != null) + { + break; + } + else + { + selKey = null; + } + } + + if (selKey != null) + { + try + { + // Process the selected key. + // As there is only one OP_WRITE key for a given channel, we need + // not to synchronize this processing to retain the order. + processSelectionKey(connection, socketChannel, selKey); + } + catch (IOException ex) + { + Log.msg("Error writing to channel: " + ex, false); + + // Cancel write events for this channel + selKey.cancel(); + connection.shutdownInput(); + connection.shutdownOutput(); + } + } + + // Eventually wait for a register operation + synchronized(NNTPDaemon.RegisterGate) { /* do nothing */ } + } + catch(CancelledKeyException ex) + { + Log.msg("ChannelWriter.run(): " + ex, true); + } + catch(Exception ex) + { + ex.printStackTrace(); + } + } // while(isRunning()) + } + + private void processSelectionKey(final NNTPConnection connection, + final SocketChannel socketChannel, final SelectionKey selKey) + throws InterruptedException, IOException + { + assert connection != null; + assert socketChannel != null; + assert selKey != null; + assert selKey.isWritable(); + + // SocketChannel is ready for writing + if(selKey.isValid()) + { + // Lock the socket channel + synchronized(socketChannel) + { + // Get next output buffer + ByteBuffer buf = connection.getOutputBuffer(); + if(buf == null) + { + // Currently we have nothing to write, so we stop the writeable + // events until we have something to write to the socket channel + //selKey.cancel(); + selKey.interestOps(0); + return; + } + + while(buf != null) // There is data to be send + { + // Write buffer to socket channel; this method does not block + if(socketChannel.write(buf) <= 0) + { + // Perhaps there is data to be written, but the SocketChannel's + // buffer is full, so we stop writing to until the next event. + break; + } + else + { + // Retrieve next buffer if available; method may return the same + // buffer instance if it still have some bytes remaining + buf = connection.getOutputBuffer(); + } + } + } + } + else + { + Log.msg("Invalid OP_WRITE key: " + selKey, false); + + if (socketChannel.socket().isClosed()) + { + connection.shutdownInput(); + connection.shutdownOutput(); + socketChannel.close(); + Log.msg("Connection closed.", true); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/Config.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/Config.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,149 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.sql.SQLException; +import org.sonews.daemon.storage.Database; +import org.sonews.util.AbstractConfig; +import org.sonews.util.TimeoutMap; + +/** + * Provides access to the program wide configuration that is stored within + * the server's database. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Config extends AbstractConfig +{ + + /** Config key constant. Value is the maximum article size in kilobytes. */ + public static final String ARTICLE_MAXSIZE = "sonews.article.maxsize"; + + /** Config key constant. Value: Amount of news that are feeded per run. */ + public static final String FEED_NEWSPERRUN = "sonews.feed.newsperrun"; + public static final String FEED_PULLINTERVAL = "sonews.feed.pullinterval"; + public static final String HOSTNAME = "sonews.hostname"; + public static final String PORT = "sonews.port"; + public static final String TIMEOUT = "sonews.timeout"; + public static final String MLPOLL_DELETEUNKNOWN = "sonews.mlpoll.deleteunknown"; + public static final String MLPOLL_HOST = "sonews.mlpoll.host"; + public static final String MLPOLL_PASSWORD = "sonews.mlpoll.password"; + public static final String MLPOLL_USER = "sonews.mlpoll.user"; + public static final String MLSEND_ADDRESS = "sonews.mlsend.address"; + public static final String MLSEND_RW_FROM = "sonews.mlsend.rewrite.from"; + public static final String MLSEND_RW_SENDER = "sonews.mlsend.rewrite.sender"; + public static final String MLSEND_HOST = "sonews.mlsend.host"; + public static final String MLSEND_PASSWORD = "sonews.mlsend.password"; + public static final String MLSEND_PORT = "sonews.mlsend.port"; + public static final String MLSEND_USER = "sonews.mlsend.user"; + + public static final String[] AVAILABLE_KEYS = { + Config.ARTICLE_MAXSIZE, + Config.FEED_NEWSPERRUN, + Config.FEED_PULLINTERVAL, + Config.HOSTNAME, + Config.MLPOLL_DELETEUNKNOWN, + Config.MLPOLL_HOST, + Config.MLPOLL_PASSWORD, + Config.MLPOLL_USER, + Config.MLSEND_ADDRESS, + Config.MLSEND_HOST, + Config.MLSEND_PASSWORD, + Config.MLSEND_PORT, + Config.MLSEND_RW_FROM, + Config.MLSEND_RW_SENDER, + Config.MLSEND_USER, + Config.PORT, + Config.TIMEOUT + }; + + private static Config instance = new Config(); + + public static Config getInstance() + { + return instance; + } + + private final TimeoutMap values + = new TimeoutMap(); + + private Config() + { + super(); + } + + /** + * Returns the config value for the given key or the defaultValue if the + * key is not found in config. + * @param key + * @param defaultValue + * @return + */ + public String get(String key, String defaultValue) + { + try + { + String configValue = values.get(key); + if(configValue == null) + { + configValue = Database.getInstance().getConfigValue(key); + if(configValue == null) + { + return defaultValue; + } + else + { + values.put(key, configValue); + return configValue; + } + } + else + { + return configValue; + } + } + catch(SQLException ex) + { + Log.msg(ex.getMessage(), false); + return defaultValue; + } + } + + /** + * Sets the config value which is identified by the given key. + * @param key + * @param value + */ + public void set(String key, String value) + { + values.put(key, value); + + try + { + // Write values to database + Database.getInstance().setConfigValue(key, value); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/ConnectionWorker.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/ConnectionWorker.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,102 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.nio.ByteBuffer; +import java.nio.channels.SocketChannel; +import java.util.concurrent.ArrayBlockingQueue; + +/** + * Does most of the work: parsing input, talking to client and Database. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class ConnectionWorker extends AbstractDaemon +{ + + // 256 pending events should be enough + private static ArrayBlockingQueue pendingChannels + = new ArrayBlockingQueue(256, true); + + /** + * Registers the given channel for further event processing. + * @param channel + */ + public static void addChannel(SocketChannel channel) + throws InterruptedException + { + pendingChannels.put(channel); + } + + /** + * Processing loop. + */ + @Override + public void run() + { + while(isRunning()) + { + try + { + // Retrieve and remove if available, otherwise wait. + SocketChannel channel = pendingChannels.take(); + + if(channel != null) + { + // Connections.getInstance().get() MAY return null + NNTPConnection conn = Connections.getInstance().get(channel); + + // Try to lock the connection object + if(conn != null && conn.tryReadLock()) + { + ByteBuffer buf = conn.getBuffers().nextInputLine(); + while(buf != null) // Complete line was received + { + final byte[] line = new byte[buf.limit()]; + buf.get(line); + ChannelLineBuffers.recycleBuffer(buf); + + // Here is the actual work done + conn.lineReceived(line); + + // Read next line as we could have already received the next line + buf = conn.getBuffers().nextInputLine(); + } + conn.unlockReadLock(); + } + else + { + addChannel(channel); + } + } + } + catch(InterruptedException ex) + { + Log.msg("ConnectionWorker interrupted: " + ex, true); + } + catch(Exception ex) + { + Log.msg("Exception in ConnectionWorker: " + ex, false); + ex.printStackTrace(); + } + } // end while(isRunning()) + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/Connections.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/Connections.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,176 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.io.IOException; +import java.net.InetSocketAddress; +import java.net.Socket; +import java.nio.channels.SocketChannel; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.ListIterator; +import java.util.Map; +import org.sonews.util.Stats; + +/** + * Daemon thread collecting all NNTPConnection instances. The thread + * checks periodically if there are stale/timed out connections and + * removes and purges them properly. + * @author Christian Lins + * @since sonews/0.5.0 + */ +final class Connections extends AbstractDaemon +{ + + private static final Connections instance = new Connections(); + + /** + * @return Active Connections instance. + */ + public static Connections getInstance() + { + return Connections.instance; + } + + private final List connections + = new ArrayList(); + private final Map connByChannel + = new HashMap(); + + private Connections() + { + setName("Connections"); + } + + /** + * Adds the given NNTPConnection to the Connections management. + * @param conn + * @see org.sonews.daemon.NNTPConnection + */ + public void add(final NNTPConnection conn) + { + synchronized(this.connections) + { + this.connections.add(conn); + this.connByChannel.put(conn.getChannel(), conn); + } + } + + /** + * @param channel + * @return NNTPConnection instance that is associated with the given + * SocketChannel. + */ + public NNTPConnection get(final SocketChannel channel) + { + synchronized(this.connections) + { + return this.connByChannel.get(channel); + } + } + + int getConnectionCount(String remote) + { + int cnt = 0; + synchronized(this.connections) + { + for(NNTPConnection conn : this.connections) + { + assert conn != null; + assert conn.getChannel() != null; + + Socket socket = conn.getChannel().socket(); + if(socket != null) + { + InetSocketAddress sockAddr = (InetSocketAddress)socket.getRemoteSocketAddress(); + if(sockAddr != null) + { + if(sockAddr.getHostName().equals(remote)) + { + cnt++; + } + } + } // if(socket != null) + } + } + return cnt; + } + + /** + * Run loops. Checks periodically for timed out connections and purged them + * from the lists. + */ + @Override + public void run() + { + while(isRunning()) + { + int timeoutMillis = 1000 * Config.getInstance().get(Config.TIMEOUT, 180); + + synchronized (this.connections) + { + final ListIterator iter = this.connections.listIterator(); + NNTPConnection conn; + + while (iter.hasNext()) + { + conn = iter.next(); + if((System.currentTimeMillis() - conn.getLastActivity()) > timeoutMillis) + { + // A connection timeout has occurred so purge the connection + iter.remove(); + + // Close and remove the channel + SocketChannel channel = conn.getChannel(); + connByChannel.remove(channel); + + try + { + // Close the channel; implicitely cancels all selectionkeys + channel.close(); + Log.msg("Disconnected: " + channel.socket().getRemoteSocketAddress() + + " (timeout)", true); + } + catch(IOException ex) + { + Log.msg("Connections.run(): " + ex, false); + } + + // Recycle the used buffers + conn.getBuffers().recycleBuffers(); + + Stats.getInstance().clientDisconnect(); + } + } + } + + try + { + Thread.sleep(10000); // Sleep ten seconds + } + catch(InterruptedException ex) + { + Log.msg("Connections Thread was interrupted: " + ex.getMessage(), false); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/LineEncoder.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/LineEncoder.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,80 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.nio.ByteBuffer; +import java.nio.CharBuffer; +import java.nio.channels.ClosedChannelException; +import java.nio.charset.Charset; +import java.nio.charset.CharsetEncoder; +import java.nio.charset.CoderResult; + +/** + * Encodes a line to buffers using the correct charset. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class LineEncoder +{ + + private CharBuffer characters; + private Charset charset; + + /** + * Constructs new LineEncoder. + * @param characters + * @param charset + */ + public LineEncoder(CharBuffer characters, Charset charset) + { + this.characters = characters; + this.charset = charset; + } + + /** + * Encodes the characters of this instance to the given ChannelLineBuffers + * using the Charset of this instance. + * @param buffer + * @throws java.nio.channels.ClosedChannelException + */ + public void encode(ChannelLineBuffers buffer) + throws ClosedChannelException + { + CharsetEncoder encoder = charset.newEncoder(); + while (characters.hasRemaining()) + { + ByteBuffer buf = ChannelLineBuffers.newLineBuffer(); + assert buf.position() == 0; + assert buf.capacity() >= 512; + + CoderResult res = encoder.encode(characters, buf, true); + + // Set limit to current position and current position to 0; + // means make ready for read from buffer + buf.flip(); + buffer.addOutputBuffer(buf); + + if (res.isUnderflow()) // All input processed + { + break; + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/Main.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/Main.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,160 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.sql.Driver; +import java.sql.DriverManager; +import java.sql.SQLException; +import java.util.Enumeration; +import java.util.Date; +import org.sonews.feed.FeedManager; +import org.sonews.mlgw.MailPoller; +import org.sonews.daemon.storage.Database; +import org.sonews.util.Log; +import org.sonews.util.io.Resource; + +/** + * Startup class of the daemon. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Main +{ + + private Main() + { + } + + /** Version information of the sonews daemon */ + public static final String VERSION = "sonews/0.5.0"; + public static final Date STARTDATE = new Date(); + + /** + * The main entrypoint. + * @param args + * @throws Exception + */ + public static void main(String[] args) throws Exception + { + System.out.println(VERSION); + Thread.currentThread().setName("Mainthread"); + + // Command line arguments + boolean feed = false; // Enable feeding? + boolean mlgw = false; // Enable Mailinglist gateway? + int port = -1; + + for(int n = 0; n < args.length; n++) + { + if(args[n].equals("-c") || args[n].equals("-config")) + { + BootstrapConfig.FILE = args[++n]; + System.out.println("Using config file " + args[n]); + } + else if(args[n].equals("-dumpjdbcdriver")) + { + System.out.println("Available JDBC drivers:"); + Enumeration drvs = DriverManager.getDrivers(); + while(drvs.hasMoreElements()) + { + System.out.println(drvs.nextElement()); + } + return; + } + else if(args[n].equals("-feed")) + { + feed = true; + } + else if(args[n].equals("-h") || args[n].equals("-help")) + { + printArguments(); + return; + } + else if(args[n].equals("-mlgw")) + { + mlgw = true; + } + else if(args[n].equals("-p")) + { + port = Integer.parseInt(args[++n]); + } + } + + // Try to load the Database; + // Do NOT USE Config or Log classes before this point because they require + // a working Database connection. + try + { + Database.getInstance(); + + // Make sure some elementary groups are existing + if(!Database.getInstance().isGroupExisting("control")) + { + Database.getInstance().addGroup("control", 0); + Log.msg("Group 'control' created.", true); + } + } + catch(SQLException ex) + { + ex.printStackTrace(); + System.err.println("Database initialization failed with " + ex.toString()); + System.err.println("Make sure you have specified the correct database" + + " settings in sonews.conf!"); + return; + } + + ChannelLineBuffers.allocateDirect(); + + // Add shutdown hook + Runtime.getRuntime().addShutdownHook(new ShutdownHook()); + + // Start the listening daemon + if(port <= 0) + { + port = Config.getInstance().get(Config.PORT, 119); + } + final NNTPDaemon daemon = NNTPDaemon.createInstance(port); + daemon.start(); + + // Start Connections purger thread... + Connections.getInstance().start(); + + // Start mailinglist gateway... + if(mlgw) + { + new MailPoller().start(); + } + + // Start feeds + if(feed) + { + FeedManager.startFeeding(); + } + + // Wait for main thread to exit (setDaemon(false)) + daemon.join(); + } + + private static void printArguments() + { + String usage = Resource.getAsString("helpers/usage", true); + System.out.println(usage); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/NNTPConnection.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/NNTPConnection.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,480 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.io.IOException; +import java.net.InetSocketAddress; +import java.nio.ByteBuffer; +import java.nio.CharBuffer; +import java.nio.channels.ClosedChannelException; +import java.nio.channels.SelectionKey; +import java.nio.channels.SocketChannel; +import java.nio.charset.Charset; +import java.util.Timer; +import java.util.TimerTask; +import org.sonews.daemon.command.ArticleCommand; +import org.sonews.daemon.command.CapabilitiesCommand; +import org.sonews.daemon.command.AbstractCommand; +import org.sonews.daemon.command.GroupCommand; +import org.sonews.daemon.command.HelpCommand; +import org.sonews.daemon.command.ListCommand; +import org.sonews.daemon.command.ListGroupCommand; +import org.sonews.daemon.command.ModeReaderCommand; +import org.sonews.daemon.command.NewGroupsCommand; +import org.sonews.daemon.command.NextPrevCommand; +import org.sonews.daemon.command.OverCommand; +import org.sonews.daemon.command.PostCommand; +import org.sonews.daemon.command.QuitCommand; +import org.sonews.daemon.command.StatCommand; +import org.sonews.daemon.command.UnsupportedCommand; +import org.sonews.daemon.command.XDaemonCommand; +import org.sonews.daemon.command.XPatCommand; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.Group; +import org.sonews.util.Stats; + +/** + * For every SocketChannel (so TCP/IP connection) there is an instance of + * this class. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class NNTPConnection +{ + + public static final String NEWLINE = "\r\n"; // RFC defines this as newline + public static final String MESSAGE_ID_PATTERN = "<[^>]+>"; + + private static final Timer cancelTimer = new Timer(true); // Thread-safe? True for run as daemon + + /** SocketChannel is generally thread-safe */ + private SocketChannel channel = null; + private Charset charset = Charset.forName("UTF-8"); + private AbstractCommand command = null; + private Article currentArticle = null; + private Group currentGroup = null; + private volatile long lastActivity = System.currentTimeMillis(); + private ChannelLineBuffers lineBuffers = new ChannelLineBuffers(); + private int readLock = 0; + private final Object readLockGate = new Object(); + private SelectionKey writeSelKey = null; + + public NNTPConnection(final SocketChannel channel) + throws IOException + { + if(channel == null) + { + throw new IllegalArgumentException("channel is null"); + } + + this.channel = channel; + Stats.getInstance().clientConnect(); + } + + /** + * Tries to get the read lock for this NNTPConnection. This method is Thread- + * safe and returns true of the read lock was successfully set. If the lock + * is still hold by another Thread the method returns false. + */ + boolean tryReadLock() + { + // As synchronizing simple types may cause deadlocks, + // we use a gate object. + synchronized(readLockGate) + { + if(readLock != 0) + { + return false; + } + else + { + readLock = Thread.currentThread().hashCode(); + return true; + } + } + } + + /** + * Releases the read lock in a Thread-safe way. + * @throws IllegalMonitorStateException if a Thread not holding the lock + * tries to release it. + */ + void unlockReadLock() + { + synchronized(readLockGate) + { + if(readLock == Thread.currentThread().hashCode()) + { + readLock = 0; + } + else + { + throw new IllegalMonitorStateException(); + } + } + } + + /** + * @return Current input buffer of this NNTPConnection instance. + */ + public ByteBuffer getInputBuffer() + { + return this.lineBuffers.getInputBuffer(); + } + + /** + * @return Output buffer of this NNTPConnection which has at least one byte + * free storage. + */ + public ByteBuffer getOutputBuffer() + { + return this.lineBuffers.getOutputBuffer(); + } + + /** + * @return ChannelLineBuffers instance associated with this NNTPConnection. + */ + public ChannelLineBuffers getBuffers() + { + return this.lineBuffers; + } + + /** + * @return true if this connection comes from a local remote address. + */ + public boolean isLocalConnection() + { + return ((InetSocketAddress)this.channel.socket().getRemoteSocketAddress()) + .getHostName().equalsIgnoreCase("localhost"); + } + + void setWriteSelectionKey(SelectionKey selKey) + { + this.writeSelKey = selKey; + } + + public void shutdownInput() + { + try + { + // Closes the input line of the channel's socket, so no new data + // will be received and a timeout can be triggered. + this.channel.socket().shutdownInput(); + } + catch(IOException ex) + { + Log.msg("Exception in NNTPConnection.shutdownInput(): " + ex, false); + if(Log.isDebug()) + { + ex.printStackTrace(); + } + } + } + + public void shutdownOutput() + { + cancelTimer.schedule(new TimerTask() + { + @Override + public void run() + { + try + { + // Closes the output line of the channel's socket. + channel.socket().shutdownOutput(); + channel.close(); + } + catch(Exception ex) + { + Log.msg("NNTPConnection.shutdownOutput(): " + ex, false); + if(Log.isDebug()) + { + ex.printStackTrace(); + } + } + } + }, 3000); + } + + public SocketChannel getChannel() + { + return this.channel; + } + + public Article getCurrentArticle() + { + return this.currentArticle; + } + + public Charset getCurrentCharset() + { + return this.charset; + } + + public Group getCurrentGroup() + { + return this.currentGroup; + } + + public void setCurrentArticle(final Article article) + { + this.currentArticle = article; + } + + public void setCurrentGroup(final Group group) + { + this.currentGroup = group; + } + + public long getLastActivity() + { + return this.lastActivity; + } + + /** + * Due to the readLockGate there is no need to synchronize this method. + * @param raw + * @throws IllegalArgumentException if raw is null. + * @throws IllegalStateException if calling thread does not own the readLock. + */ + void lineReceived(byte[] raw) + { + if(raw == null) + { + throw new IllegalArgumentException("raw is null"); + } + + if(readLock == 0 || readLock != Thread.currentThread().hashCode()) + { + throw new IllegalStateException("readLock not properly set"); + } + + this.lastActivity = System.currentTimeMillis(); + + String line = new String(raw, this.charset); + + // There might be a trailing \r, but trim() is a bad idea + // as it removes also leading spaces from long header lines. + if(line.endsWith("\r")) + { + line = line.substring(0, line.length() - 1); + } + + Log.msg("<< " + line, true); + + if(command == null) + { + command = parseCommandLine(line); + assert command != null; + } + + try + { + // The command object will process the line we just received + command.processLine(line); + } + catch(ClosedChannelException ex0) + { + try + { + Log.msg("Connection to " + channel.socket().getRemoteSocketAddress() + + " closed: " + ex0, true); + } + catch(Exception ex0a) + { + ex0a.printStackTrace(); + } + } + catch(Exception ex1) + { + try + { + command = null; + ex1.printStackTrace(); + println("500 Internal server error"); + } + catch(Exception ex2) + { + ex2.printStackTrace(); + } + } + + if(command == null || command.hasFinished()) + { + command = null; + charset = Charset.forName("UTF-8"); // Reset to default + } + } + + /** + * This method performes several if/elseif constructs to determine the + * fitting command object. + * TODO: This string comparisons are probably slow! + * @param line + * @return + */ + private AbstractCommand parseCommandLine(String line) + { + AbstractCommand cmd = new UnsupportedCommand(this); + String cmdStr = line.split(" ")[0]; + + if(cmdStr.equalsIgnoreCase("ARTICLE") || + cmdStr.equalsIgnoreCase("BODY")) + { + cmd = new ArticleCommand(this); + } + else if(cmdStr.equalsIgnoreCase("CAPABILITIES")) + { + cmd = new CapabilitiesCommand(this); + } + else if(cmdStr.equalsIgnoreCase("GROUP")) + { + cmd = new GroupCommand(this); + } + else if(cmdStr.equalsIgnoreCase("HEAD")) + { + cmd = new ArticleCommand(this); + } + else if(cmdStr.equalsIgnoreCase("HELP")) + { + cmd = new HelpCommand(this); + } + else if(cmdStr.equalsIgnoreCase("LIST")) + { + cmd = new ListCommand(this); + } + else if(cmdStr.equalsIgnoreCase("LISTGROUP")) + { + cmd = new ListGroupCommand(this); + } + else if(cmdStr.equalsIgnoreCase("MODE")) + { + cmd = new ModeReaderCommand(this); + } + else if(cmdStr.equalsIgnoreCase("NEWGROUPS")) + { + cmd = new NewGroupsCommand(this); + } + else if(cmdStr.equalsIgnoreCase("NEXT") || + cmdStr.equalsIgnoreCase("PREV")) + { + cmd = new NextPrevCommand(this); + } + else if(cmdStr.equalsIgnoreCase("OVER") || + cmdStr.equalsIgnoreCase("XOVER")) // for compatibility with older RFCs + { + cmd = new OverCommand(this); + } + else if(cmdStr.equalsIgnoreCase("POST")) + { + cmd = new PostCommand(this); + } + else if(cmdStr.equalsIgnoreCase("QUIT")) + { + cmd = new QuitCommand(this); + } + else if(cmdStr.equalsIgnoreCase("STAT")) + { + cmd = new StatCommand(this); + } + else if(cmdStr.equalsIgnoreCase("XDAEMON")) + { + cmd = new XDaemonCommand(this); + } + else if(cmdStr.equalsIgnoreCase("XPAT")) + { + cmd = new XPatCommand(this); + } + + return cmd; + } + + /** + * Puts the given line into the output buffer, adds a newline character + * and returns. The method returns immediately and does not block until + * the line was sent. If line is longer than 510 octets it is split up in + * several lines. Each line is terminated by \r\n (NNTPConnection.NEWLINE). + * @param line + */ + public void println(final CharSequence line, final Charset charset) + throws IOException + { + writeToChannel(CharBuffer.wrap(line), charset, line); + writeToChannel(CharBuffer.wrap(NEWLINE), charset, null); + } + + /** + * Encodes the given CharBuffer using the given Charset to a bunch of + * ByteBuffers (each 512 bytes large) and enqueues them for writing at the + * connected SocketChannel. + * @throws java.io.IOException + */ + private void writeToChannel(CharBuffer characters, final Charset charset, + CharSequence debugLine) + throws IOException + { + if(!charset.canEncode()) + { + Log.msg("FATAL: Charset " + charset + " cannot encode!", false); + return; + } + + // Write characters to output buffers + LineEncoder lenc = new LineEncoder(characters, charset); + lenc.encode(lineBuffers); + + // Enable OP_WRITE events so that the buffers are processed + try + { + this.writeSelKey.interestOps(SelectionKey.OP_WRITE); + ChannelWriter.getInstance().getSelector().wakeup(); + } + catch (Exception ex) // CancelledKeyException and ChannelCloseException + { + Log.msg("NNTPConnection.writeToChannel(): " + ex, false); + return; + } + + // Update last activity timestamp + this.lastActivity = System.currentTimeMillis(); + if(debugLine != null) + { + Log.msg(">> " + debugLine, true); + } + } + + public void println(final CharSequence line) + throws IOException + { + println(line, charset); + } + + public void print(final String line) + throws IOException + { + writeToChannel(CharBuffer.wrap(line), charset, line); + } + + public void setCurrentCharset(final Charset charset) + { + this.charset = charset; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/NNTPDaemon.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/NNTPDaemon.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,195 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import org.sonews.util.Log; +import java.io.IOException; +import java.net.BindException; +import java.net.InetSocketAddress; +import java.net.ServerSocket; +import java.nio.channels.CancelledKeyException; +import java.nio.channels.ClosedChannelException; +import java.nio.channels.SelectionKey; +import java.nio.channels.Selector; +import java.nio.channels.ServerSocketChannel; +import java.nio.channels.SocketChannel; + +/** + * NNTP daemon using SelectableChannels. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class NNTPDaemon extends AbstractDaemon +{ + + public static final Object RegisterGate = new Object(); + + private static NNTPDaemon instance = null; + + public static synchronized NNTPDaemon createInstance(int port) + { + if(instance == null) + { + instance = new NNTPDaemon(port); + return instance; + } + else + { + throw new RuntimeException("NNTPDaemon.createInstance() called twice"); + } + } + + private int port; + + private NNTPDaemon(final int port) + { + Log.msg("Server listening on port " + port, false); + this.port = port; + } + + @Override + public void run() + { + try + { + // Create a Selector that handles the SocketChannel multiplexing + final Selector readSelector = Selector.open(); + final Selector writeSelector = Selector.open(); + + // Start working threads + final int workerThreads = Runtime.getRuntime().availableProcessors() * 4; + ConnectionWorker[] cworkers = new ConnectionWorker[workerThreads]; + for(int n = 0; n < workerThreads; n++) + { + cworkers[n] = new ConnectionWorker(); + cworkers[n].start(); + } + + ChannelWriter.getInstance().setSelector(writeSelector); + ChannelReader.getInstance().setSelector(readSelector); + ChannelWriter.getInstance().start(); + ChannelReader.getInstance().start(); + + final ServerSocketChannel serverSocketChannel = ServerSocketChannel.open(); + serverSocketChannel.configureBlocking(true); // Set to blocking mode + + // Configure ServerSocket; bind to socket... + final ServerSocket serverSocket = serverSocketChannel.socket(); + serverSocket.bind(new InetSocketAddress(this.port)); + + while(isRunning()) + { + SocketChannel socketChannel; + + try + { + // As we set the server socket channel to blocking mode the accept() + // method will block. + socketChannel = serverSocketChannel.accept(); + socketChannel.configureBlocking(false); + assert socketChannel.isConnected(); + assert socketChannel.finishConnect(); + } + catch(IOException ex) + { + // Under heavy load an IOException "Too many open files may + // be thrown. It most cases we should slow down the connection + // accepting, to give the worker threads some time to process work. + Log.msg("IOException while accepting connection: " + ex.getMessage(), false); + Log.msg("Connection accepting sleeping for seconds...", true); + Thread.sleep(5000); // 5 seconds + continue; + } + + final NNTPConnection conn; + try + { + conn = new NNTPConnection(socketChannel); + Connections.getInstance().add(conn); + } + catch(IOException ex) + { + Log.msg(ex.getLocalizedMessage(), false); + socketChannel.close(); + continue; + } + + try + { + SelectionKey selKeyWrite = + registerSelector(writeSelector, socketChannel, SelectionKey.OP_WRITE); + registerSelector(readSelector, socketChannel, SelectionKey.OP_READ); + + Log.msg("Connected: " + socketChannel.socket().getRemoteSocketAddress(), true); + + // Set write selection key and send hello to client + conn.setWriteSelectionKey(selKeyWrite); + conn.println("200 " + Config.getInstance().get(Config.HOSTNAME, "localhost") + + " " + Main.VERSION + " news server ready - (posting ok)."); + } + catch(CancelledKeyException cke) + { + Log.msg("CancelledKeyException " + cke.getMessage() + " was thrown: " + + socketChannel.socket(), false); + } + catch(ClosedChannelException cce) + { + Log.msg("ClosedChannelException " + cce.getMessage() + " was thrown: " + + socketChannel.socket(), false); + } + } + } + catch(BindException ex) + { + // Could not bind to socket; this is a fatal problem; so perform shutdown + ex.printStackTrace(); + System.exit(1); + } + catch(IOException ex) + { + ex.printStackTrace(); + } + catch(Exception ex) + { + ex.printStackTrace(); + } + } + + public static SelectionKey registerSelector(final Selector selector, + final SocketChannel channel, final int op) + throws CancelledKeyException, ClosedChannelException + { + // Register the selector at the channel, so that it will be notified + // on the socket's events + synchronized(RegisterGate) + { + // Wakeup the currently blocking reader/writer thread; we have locked + // the RegisterGate to prevent the awakened thread to block again + selector.wakeup(); + + // Lock the selector to prevent the waiting worker threads going into + // selector.select() which would block the selector. + synchronized (selector) + { + return channel.register(selector, op, null); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/ShutdownHook.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/ShutdownHook.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,83 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.sql.SQLException; +import java.util.Map; + +/** + * Will force all other threads to shutdown cleanly. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class ShutdownHook extends Thread +{ + + /** + * Called when the JVM exits. + */ + @Override + public void run() + { + System.out.println("sonews: Trying to shutdown all threads..."); + + Map threadsMap = Thread.getAllStackTraces(); + for(Thread thread : threadsMap.keySet()) + { + // Interrupt the thread if it's a AbstractDaemon + AbstractDaemon daemon; + if(thread instanceof AbstractDaemon && thread.isAlive()) + { + try + { + daemon = (AbstractDaemon)thread; + daemon.shutdownNow(); + } + catch(SQLException ex) + { + System.out.println("sonews: " + ex); + } + } + } + + for(Thread thread : threadsMap.keySet()) + { + AbstractDaemon daemon; + if(thread instanceof AbstractDaemon && thread.isAlive()) + { + daemon = (AbstractDaemon)thread; + System.out.println("sonews: Waiting for " + daemon + " to exit..."); + try + { + daemon.join(500); + } + catch(InterruptedException ex) + { + System.out.println(ex.getLocalizedMessage()); + } + } + } + + // We have notified all not-sleeping AbstractDaemons of the shutdown; + // all other threads can be simply purged on VM shutdown + + System.out.println("sonews: Clean shutdown."); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/AbstractCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/AbstractCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,87 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.nio.charset.Charset; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.Group; + +/** + * Base class for all command handling classes. + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public abstract class AbstractCommand +{ + + protected NNTPConnection connection; + + public AbstractCommand(final NNTPConnection connection) + { + this.connection = connection; + } + + protected Article getCurrentArticle() + { + return connection.getCurrentArticle(); + } + + protected Group getCurrentGroup() + { + return connection.getCurrentGroup(); + } + + protected void setCurrentArticle(final Article current) + { + connection.setCurrentArticle(current); + } + + protected void setCurrentGroup(final Group current) + { + connection.setCurrentGroup(current); + } + + public abstract void processLine(String line) + throws IOException, SQLException; + + protected void println(final String line) + throws IOException + { + connection.println(line); + } + + protected void println(final String line, final Charset charset) + throws IOException + { + connection.println(line, charset); + } + + protected void printStatus(final int status, final String msg) + throws IOException + { + println(status + " " + msg); + } + + public abstract boolean hasFinished(); + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/ArticleCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/ArticleCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,164 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Group; + +/** + * Class handling the ARTICLE, BODY and HEAD commands. + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public class ArticleCommand extends AbstractCommand +{ + + public ArticleCommand(final NNTPConnection connection) + { + super(connection); + } + + @Override + public boolean hasFinished() + { + return true; + } + + // TODO: Refactor this method to reduce its complexity! + @Override + public void processLine(final String line) + throws IOException + { + final String[] command = line.split(" "); + + Article article = null; + long artIndex = -1; + if (command.length == 1) + { + article = getCurrentArticle(); + if (article == null) + { + printStatus(420, "no current article has been selected"); + return; + } + } + else if (command[1].matches(NNTPConnection.MESSAGE_ID_PATTERN)) + { + // Message-ID + article = Article.getByMessageID(command[1]); + if (article == null) + { + printStatus(430, "no such article found"); + return; + } + } + else + { + // Message Number + try + { + Group currentGroup = connection.getCurrentGroup(); + if(currentGroup == null) + { + printStatus(400, "no group selected"); + return; + } + + artIndex = Long.parseLong(command[1]); + article = Article.getByArticleNumber(artIndex, currentGroup); + } + catch(NumberFormatException ex) + { + ex.printStackTrace(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + + if (article == null) + { + printStatus(423, "no such article number in this group"); + return; + } + setCurrentArticle(article); + } + + if(command[0].equalsIgnoreCase("ARTICLE")) + { + printStatus(220, artIndex + " " + article.getMessageID() + + " article retrieved - head and body follow"); + + println(article.getHeaderSource()); + + println(""); + println(article.getBody(), article.getBodyCharset()); + println("."); + } + else if(command[0].equalsIgnoreCase("BODY")) + { + printStatus(222, artIndex + " " + article.getMessageID() + " body"); + println(article.getBody(), article.getBodyCharset()); + println("."); + } + + /* + * HEAD: This command is mandatory. + * + * Syntax + * HEAD message-id + * HEAD number + * HEAD + * + * Responses + * + * First form (message-id specified) + * 221 0|n message-id Headers follow (multi-line) + * 430 No article with that message-id + * + * Second form (article number specified) + * 221 n message-id Headers follow (multi-line) + * 412 No newsgroup selected + * 423 No article with that number + * + * Third form (current article number used) + * 221 n message-id Headers follow (multi-line) + * 412 No newsgroup selected + * 420 Current article number is invalid + * + * Parameters + * number Requested article number + * n Returned article number + * message-id Article message-id + */ + else if(command[0].equalsIgnoreCase("HEAD")) + { + printStatus(221, artIndex + " " + article.getMessageID() + + " Headers follow (multi-line)"); + + println(article.getHeaderSource()); + println("."); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/CapabilitiesCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/CapabilitiesCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,80 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import org.sonews.daemon.NNTPConnection; + +/** + * 
+ * The CAPABILITIES command allows a client to determine the
+ * capabilities of the server at any given time.
+ *
+ * This command MAY be issued at any time; the server MUST NOT require
+ * it to be issued in order to make use of any capability. The response
+ * generated by this command MAY change during a session because of
+ * other state information (which, in turn, may be changed by the
+ * effects of other commands or by external events).  An NNTP client is
+ * only able to get the current and correct information concerning
+ * available capabilities at any point during a session by issuing a
+ * CAPABILITIES command at that point of that session and processing the
+ * response.
+ *
+ * @author Christian Lins + * @since sonews/0.5.0 + */ +public class CapabilitiesCommand extends AbstractCommand +{ + + protected static final String[] CAPABILITIES = new String[] + { + "VERSION 2", // MUST be the first one; VERSION 2 refers to RFC3977 + "READER", // Server implements commands for reading + "POST", // Server implements POST command + "OVER" // Server implements OVER command + }; + + public CapabilitiesCommand(final NNTPConnection conn) + { + super(conn); + } + + /** + * First called after one call to processLine(). + * @return + */ + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException + { + printStatus(101, "Capabilities list:"); + for(String cap : CAPABILITIES) + { + println(cap); + } + println("."); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/GroupCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/GroupCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,90 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Group; + +/** + * Class handling the GROUP command. + * 
+ *  Syntax
+ *    GROUP group
+ *
+ *  Responses
+ *    211 number low high group     Group successfully selected
+ *    411                           No such newsgroup
+ *
+ *  Parameters
+ *    group     Name of newsgroup
+ *    number    Estimated number of articles in the group
+ *    low       Reported low water mark
+ *    high      Reported high water mark
+ *
+ * (from RFC 3977) + * + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public class GroupCommand extends AbstractCommand +{ + + public GroupCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + final String[] command = line.split(" "); + + Group group; + if(command.length >= 2) + { + group = Group.getByName(command[1]); + if(group == null) + { + printStatus(411, "no such news group"); + } + else + { + setCurrentGroup(group); + + printStatus(211, group.getPostingsCount() + " " + group.getFirstArticleNumber() + + " " + group.getLastArticleNumber() + " " + group.getName() + " group selected"); + } + } + else + { + printStatus(500, "no group name given"); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/HelpCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/HelpCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,63 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import org.sonews.daemon.NNTPConnection; +import org.sonews.util.io.Resource; + +/** + * This command provides a short summary of the commands that are + * understood by this implementation of the server. The help text will + * be presented as a multi-line data block following the 100 response + * code (taken from RFC). + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class HelpCommand extends AbstractCommand +{ + + public HelpCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException + { + printStatus(100, "help text follows"); + + final String[] help = Resource + .getAsString("helpers/helptext", true).split("\n"); + for(String hstr : help) + { + println(hstr); + } + + println("."); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/ListCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/ListCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,109 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Group; + +/** + * Class handling the LIST command. + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public class ListCommand extends AbstractCommand +{ + + public ListCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + final String[] command = line.split(" "); + + if (command.length >= 2) + { + if (command[1].equalsIgnoreCase("OVERVIEW.FMT")) + { + printStatus(215, "information follows"); + println("Subject:\nFrom:\nDate:\nMessage-ID:\nReferences:\nBytes:\nLines:\nXref"); + println("."); + } + else if (command[1].equalsIgnoreCase("NEWSGROUPS")) + { + printStatus(215, "information follows"); + final List list = Group.getAll(); + for (Group g : list) + { + println(g.getName() + "\t" + "-"); + } + println("."); + } + else if (command[1].equalsIgnoreCase("SUBSCRIPTIONS")) + { + printStatus(215, "information follows"); + println("."); + } + else if (command[1].equalsIgnoreCase("EXTENSIONS")) + { + printStatus(202, "Supported NNTP extensions."); + println("LISTGROUP"); + println("."); + + } + else + { + printStatus(500, "unknown argument to LIST command"); + } + } + else + { + final List groups = Group.getAll(); + if(groups != null) + { + printStatus(215, "list of newsgroups follows"); + for (Group g : groups) + { + // Indeed first the higher article number then the lower + println(g.getName() + " " + g.getLastArticleNumber() + " " + + g.getFirstArticleNumber() + " y"); + } + println("."); + } + else + { + printStatus(500, "server database malfunction"); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/ListGroupCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/ListGroupCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,81 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Group; + +/** + * Class handling the LISTGROUP command. + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public class ListGroupCommand extends AbstractCommand +{ + + public ListGroupCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String commandName) + throws IOException, SQLException + { + final String[] command = commandName.split(" "); + + Group group; + if(command.length >= 2) + { + group = Group.getByName(command[1]); + } + else + { + group = getCurrentGroup(); + } + + if (group == null) + { + printStatus(412, "no group selected; use GROUP command"); + return; + } + + List ids = group.getArticleNumbers(); + printStatus(211, ids.size() + " " + + group.getFirstArticleNumber() + " " + + group.getLastArticleNumber() + " list of article numbers follow"); + for(long id : ids) + { + // One index number per line + println(Long.toString(id)); + } + println("."); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/ModeReaderCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/ModeReaderCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,58 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; + +/** + * Class handling the MODE READER command. This command actually does nothing + * but returning a success status code. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ModeReaderCommand extends AbstractCommand +{ + + public ModeReaderCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) throws IOException, SQLException + { + if(line.equalsIgnoreCase("MODE READER")) + { + printStatus(200, "hello you can post"); + } + else + { + printStatus(500, "I do not know this mode command"); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/NewGroupsCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/NewGroupsCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,65 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; + +/** + * Class handling the NEWGROUPS command. + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public class NewGroupsCommand extends AbstractCommand +{ + + public NewGroupsCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + final String[] command = line.split(" "); + + if(command.length == 3) + { + printStatus(231, "list of new newsgroups follows"); + + // Currently we do not store a group's creation date; + // so we return an empty list which is a valid response + println("."); + } + else + { + printStatus(500, "invalid command usage"); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/NextPrevCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/NextPrevCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,100 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.Group; + +/** + * Class handling the NEXT and LAST command. + * @author Christian Lins + * @author Dennis Schwerdel + * @since n3tpd/0.1 + */ +public class NextPrevCommand extends AbstractCommand +{ + + public NextPrevCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + final Article currA = getCurrentArticle(); + final Group currG = getCurrentGroup(); + + if (currA == null) + { + printStatus(420, "no current article has been selected"); + return; + } + + if (currG == null) + { + printStatus(412, "no newsgroup selected"); + return; + } + + final String[] command = line.split(" "); + + if(command[0].equalsIgnoreCase("NEXT")) + { + selectNewArticle(currA, currG, 1); + } + else if(command[0].equalsIgnoreCase("PREV")) + { + selectNewArticle(currA, currG, -1); + } + else + { + printStatus(500, "internal server error"); + } + } + + private void selectNewArticle(Article article, Group grp, final int delta) + throws IOException, SQLException + { + assert article != null; + + article = Article.getByArticleNumber(article.getIndexInGroup(grp) + delta, grp); + + if(article == null) + { + printStatus(421, "no next article in this group"); + } + else + { + setCurrentArticle(article); + printStatus(223, article.getIndexInGroup(getCurrentGroup()) + " " + article.getMessageID() + " article retrieved - request text separately"); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/OverCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/OverCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,281 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; +import org.sonews.util.Log; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.ArticleHead; +import org.sonews.daemon.storage.Headers; +import org.sonews.util.Pair; + +/** + * Class handling the OVER/XOVER command. + * + * Description of the XOVER command: + * 
+ * XOVER [range]
+ *
+ * The XOVER command returns information from the overview
+ * database for the article(s) specified.
+ *
+ * The optional range argument may be any of the following:
+ *              an article number
+ *              an article number followed by a dash to indicate
+ *                 all following
+ *              an article number followed by a dash followed by
+ *                 another article number
+ *
+ * If no argument is specified, then information from the
+ * current article is displayed. Successful responses start
+ * with a 224 response followed by the overview information
+ * for all matched messages. Once the output is complete, a
+ * period is sent on a line by itself. If no argument is
+ * specified, the information for the current article is
+ * returned.  A news group must have been selected earlier,
+ * else a 412 error response is returned. If no articles are
+ * in the range specified, a 420 error response is returned
+ * by the server. A 502 response will be returned if the
+ * client only has permission to transfer articles.
+ *
+ * Each line of output will be formatted with the article number,
+ * followed by each of the headers in the overview database or the
+ * article itself (when the data is not available in the overview
+ * database) for that article separated by a tab character.  The
+ * sequence of fields must be in this order: subject, author,
+ * date, message-id, references, byte count, and line count. Other
+ * optional fields may follow line count. Other optional fields may
+ * follow line count. These fields are specified by examining the
+ * response to the LIST OVERVIEW.FMT command. Where no data exists,
+ * a null field must be provided (i.e. the output will have two tab
+ * characters adjacent to each other). Servers should not output
+ * fields for articles that have been removed since the XOVER database
+ * was created.
+ *
+ * The LIST OVERVIEW.FMT command should be implemented if XOVER
+ * is implemented. A client can use LIST OVERVIEW.FMT to determine
+ * what optional fields  and in which order all fields will be
+ * supplied by the XOVER command. 
+ *
+ * Note that any tab and end-of-line characters in any header
+ * data that is returned will be converted to a space character.
+ *
+ * Responses:
+ *
+ *   224 Overview information follows
+ *   412 No news group current selected
+ *   420 No article(s) selected
+ *   502 no permission
+ *
+ * OVER defines additional responses:
+ *
+ *  First form (message-id specified)
+ *    224    Overview information follows (multi-line)
+ *    430    No article with that message-id
+ *
+ *  Second form (range specified)
+ *    224    Overview information follows (multi-line)
+ *    412    No newsgroup selected
+ *    423    No articles in that range
+ *
+ *  Third form (current article number used)
+ *    224    Overview information follows (multi-line)
+ *    412    No newsgroup selected
+ *    420    Current article number is invalid
+ *
+ *
+ * @author Christian Lins + * @since sonews/0.5.0 + */ +public class OverCommand extends AbstractCommand +{ + + public static final int MAX_LINES_PER_DBREQUEST = 100; + + public OverCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + if(getCurrentGroup() == null) + { + printStatus(412, "No news group current selected"); + } + else + { + String[] command = line.split(" "); + + // If no parameter was specified, show information about + // the currently selected article(s) + if(command.length == 1) + { + final Article art = getCurrentArticle(); + if(art == null) + { + printStatus(420, "No article(s) selected"); + return; + } + + println(buildOverview(art, -1)); + } + // otherwise print information about the specified range + else + { + int artStart; + int artEnd = getCurrentGroup().getLastArticleNumber(); + String[] nums = command[1].split("-"); + if(nums.length >= 1) + { + try + { + artStart = Integer.parseInt(nums[0]); + } + catch(NumberFormatException e) + { + Log.msg(e.getMessage(), true); + artStart = Integer.parseInt(command[1]); + } + } + else + { + artStart = getCurrentGroup().getFirstArticleNumber(); + } + + if(nums.length >=2) + { + try + { + artEnd = Integer.parseInt(nums[1]); + } + catch(NumberFormatException e) + { + e.printStackTrace(); + } + } + + if(artStart > artEnd) + { + if(command[0].equalsIgnoreCase("OVER")) + { + printStatus(423, "No articles in that range"); + } + else + { + printStatus(224, "(empty) overview information follows:"); + println("."); + } + } + else + { + for(int n = artStart; n <= artEnd; n += MAX_LINES_PER_DBREQUEST) + { + int nEnd = Math.min(n + MAX_LINES_PER_DBREQUEST - 1, artEnd); + List> articleHeads = getCurrentGroup() + .getArticleHeads(n, nEnd); + if(articleHeads.isEmpty() && n == artStart + && command[0].equalsIgnoreCase("OVER")) + { + // This reply is only valid for OVER, not for XOVER command + printStatus(423, "No articles in that range"); + return; + } + else if(n == artStart) + { + // XOVER replies this although there is no data available + printStatus(224, "Overview information follows"); + } + + for(Pair article : articleHeads) + { + String overview = buildOverview(article.getB(), article.getA()); + println(overview); + } + } // for + println("."); + } + } + } + } + + private String buildOverview(ArticleHead art, long nr) + { + StringBuilder overview = new StringBuilder(); + overview.append(nr); + overview.append('\t'); + + String subject = art.getHeader(Headers.SUBJECT)[0]; + if("".equals(subject)) + { + subject = ""; + } + overview.append(escapeString(subject)); + overview.append('\t'); + + overview.append(escapeString(art.getHeader(Headers.FROM)[0])); + overview.append('\t'); + overview.append(escapeString(art.getHeader(Headers.DATE)[0])); + overview.append('\t'); + overview.append(escapeString(art.getHeader(Headers.MESSAGE_ID)[0])); + overview.append('\t'); + overview.append(escapeString(art.getHeader(Headers.REFERENCES)[0])); + overview.append('\t'); + + String bytes = art.getHeader(Headers.BYTES)[0]; + if("".equals(bytes)) + { + bytes = "0"; + } + overview.append(escapeString(bytes)); + overview.append('\t'); + + String lines = art.getHeader(Headers.LINES)[0]; + if("".equals(lines)) + { + lines = "0"; + } + overview.append(escapeString(lines)); + overview.append('\t'); + overview.append(escapeString(art.getHeader(Headers.XREF)[0])); + + // Remove trailing tabs if some data is empty + return overview.toString().trim(); + } + + private String escapeString(String str) + { + String nstr = str.replace("\r", ""); + nstr = nstr.replace('\n', ' '); + nstr = nstr.replace('\t', ' '); + return nstr.trim(); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/PostCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/PostCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,350 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; + +import java.io.ByteArrayInputStream; +import java.nio.charset.Charset; +import java.nio.charset.IllegalCharsetNameException; +import java.nio.charset.UnsupportedCharsetException; +import java.sql.SQLException; +import java.util.Locale; +import javax.mail.MessagingException; +import javax.mail.internet.AddressException; +import javax.mail.internet.InternetHeaders; +import org.sonews.daemon.Config; +import org.sonews.util.Log; +import org.sonews.mlgw.Dispatcher; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.Database; +import org.sonews.daemon.storage.Group; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Headers; +import org.sonews.feed.FeedManager; +import org.sonews.util.Stats; + +/** + * Implementation of the POST command. This command requires multiple lines + * from the client, so the handling of asynchronous reading is a little tricky + * to handle. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class PostCommand extends AbstractCommand +{ + + private final Article article = new Article(); + private int lineCount = 0; + private long bodySize = 0; + private InternetHeaders headers = null; + private long maxBodySize = + Config.getInstance().get(Config.ARTICLE_MAXSIZE, 128) * 1024L; // Size in bytes + private PostState state = PostState.WaitForLineOne; + private final StringBuilder strBody = new StringBuilder(); + private final StringBuilder strHead = new StringBuilder(); + + public PostCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return this.state == PostState.Finished; + } + + /** + * Process the given line String. line.trim() was called by NNTPConnection. + * @param line + * @throws java.io.IOException + * @throws java.sql.SQLException + */ + @Override // TODO: Refactor this method to reduce complexity! + public void processLine(String line) + throws IOException, SQLException + { + switch(state) + { + case WaitForLineOne: + { + if(line.equalsIgnoreCase("POST")) + { + printStatus(340, "send article to be posted. End with ."); + state = PostState.ReadingHeaders; + } + else + { + printStatus(500, "invalid command usage"); + } + break; + } + case ReadingHeaders: + { + strHead.append(line); + strHead.append(NNTPConnection.NEWLINE); + + if("".equals(line) || ".".equals(line)) + { + // we finally met the blank line + // separating headers from body + + try + { + // Parse the header using the InternetHeader class from JavaMail API + headers = new InternetHeaders( + new ByteArrayInputStream(strHead.toString().trim() + .getBytes(connection.getCurrentCharset()))); + + // add the header entries for the article + article.setHeaders(headers); + } + catch (MessagingException e) + { + e.printStackTrace(); + printStatus(500, "posting failed - invalid header"); + state = PostState.Finished; + break; + } + + // Change charset for reading body; + // for multipart messages UTF-8 is returned + connection.setCurrentCharset(article.getBodyCharset()); + + state = PostState.ReadingBody; + + if(".".equals(line)) + { + // Post an article without body + postArticle(article); + state = PostState.Finished; + } + } + break; + } + case ReadingBody: + { + if(".".equals(line)) + { + // Set some headers needed for Over command + headers.setHeader(Headers.LINES, Integer.toString(lineCount)); + headers.setHeader(Headers.BYTES, Long.toString(bodySize)); + + if(strBody.length() >= 2) + { + strBody.deleteCharAt(strBody.length() - 1); // Remove last newline + strBody.deleteCharAt(strBody.length() - 1); // Remove last CR + } + article.setBody(strBody.toString()); // set the article body + + postArticle(article); + state = PostState.Finished; + } + else + { + bodySize += line.length() + 1; + lineCount++; + + // Add line to body buffer + strBody.append(line); + strBody.append(NNTPConnection.NEWLINE); + + if(bodySize > maxBodySize) + { + printStatus(500, "article is too long"); + state = PostState.Finished; + break; + } + + // Check if this message is a MIME-multipart message and needs a + // charset change + try + { + line = line.toLowerCase(Locale.ENGLISH); + if(line.startsWith(Headers.CONTENT_TYPE)) + { + int idxStart = line.indexOf("charset=") + "charset=".length(); + int idxEnd = line.indexOf(";", idxStart); + if(idxEnd < 0) + { + idxEnd = line.length(); + } + + if(idxStart > 0) + { + String charsetName = line.substring(idxStart, idxEnd); + if(charsetName.length() > 0 && charsetName.charAt(0) == '"') + { + charsetName = charsetName.substring(1, charsetName.length() - 1); + } + + try + { + connection.setCurrentCharset(Charset.forName(charsetName)); + } + catch(IllegalCharsetNameException ex) + { + Log.msg("PostCommand: " + ex, false); + } + catch(UnsupportedCharsetException ex) + { + Log.msg("PostCommand: " + ex, false); + } + } // if(idxStart > 0) + } + } + catch(Exception ex) + { + ex.printStackTrace(); + } + } + break; + } + default: + Log.msg("PostCommand::processLine(): already finished...", false); + } + } + + /** + * Article is a control message and needs special handling. + * @param article + */ + private void controlMessage(Article article) + throws IOException + { + String[] ctrl = article.getHeader(Headers.CONTROL)[0].split(" "); + if(ctrl.length == 2) // "cancel " + { + try + { + Database.getInstance().delete(ctrl[1]); + + // Move cancel message to "control" group + article.setHeader(Headers.NEWSGROUPS, "control"); + Database.getInstance().addArticle(article); + printStatus(240, "article cancelled"); + } + catch(SQLException ex) + { + Log.msg(ex, false); + printStatus(500, "internal server error"); + } + } + else + { + printStatus(441, "unknown Control header"); + } + } + + private void supersedeMessage(Article article) + throws IOException + { + try + { + String oldMsg = article.getHeader(Headers.SUPERSEDES)[0]; + Database.getInstance().delete(oldMsg); + Database.getInstance().addArticle(article); + printStatus(240, "article replaced"); + } + catch(SQLException ex) + { + Log.msg(ex, false); + printStatus(500, "internal server error"); + } + } + + private void postArticle(Article article) + throws IOException + { + if(article.getHeader(Headers.CONTROL)[0].length() > 0) + { + controlMessage(article); + } + else if(article.getHeader(Headers.SUPERSEDES)[0].length() > 0) + { + supersedeMessage(article); + } + else // Post the article regularily + { + // Try to create the article in the database or post it to + // appropriate mailing list + try + { + boolean success = false; + String[] groupnames = article.getHeader(Headers.NEWSGROUPS)[0].split(","); + for(String groupname : groupnames) + { + Group group = Database.getInstance().getGroup(groupname); + if(group != null) + { + if(group.isMailingList() && !connection.isLocalConnection()) + { + // Send to mailing list; the Dispatcher writes + // statistics to database + Dispatcher.toList(article); + success = true; + } + else + { + // Store in database + if(!Database.getInstance().isArticleExisting(article.getMessageID())) + { + Database.getInstance().addArticle(article); + + // Log this posting to statistics + Stats.getInstance().mailPosted( + article.getHeader(Headers.NEWSGROUPS)[0]); + } + success = true; + } + } + } // end for + + if(success) + { + printStatus(240, "article posted ok"); + FeedManager.queueForPush(article); + } + else + { + printStatus(441, "newsgroup not found"); + } + } + catch(AddressException ex) + { + Log.msg(ex.getMessage(), true); + printStatus(441, "invalid sender address"); + } + catch(MessagingException ex) + { + // A MessageException is thrown when the sender email address is + // invalid or something is wrong with the SMTP server. + System.err.println(ex.getLocalizedMessage()); + printStatus(441, ex.getClass().getCanonicalName() + ": " + ex.getLocalizedMessage()); + } + catch(SQLException ex) + { + ex.printStackTrace(); + printStatus(500, "internal server error"); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/PostState.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/PostState.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,29 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +/** + * States of the POST command's finite state machine. + * @author Christian Lins + * @since sonews/0.5.0 + */ +enum PostState +{ + WaitForLineOne, ReadingHeaders, ReadingBody, Finished +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/QuitCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/QuitCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,54 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; + +/** + * Implementation of the QUIT command; client wants to shutdown the connection. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class QuitCommand extends AbstractCommand +{ + + public QuitCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + printStatus(205, "cya"); + + this.connection.shutdownInput(); + this.connection.shutdownOutput(); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/StatCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/StatCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,100 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.NNTPConnection; + +/** + * Implementation of the STAT command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class StatCommand extends AbstractCommand +{ + + public StatCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + // TODO: Method has various exit points => Refactor! + @Override + public void processLine(final String line) + throws IOException, SQLException + { + final String[] command = line.split(" "); + + Article article = null; + if(command.length == 1) + { + article = getCurrentArticle(); + if(article == null) + { + printStatus(420, "no current article has been selected"); + return; + } + } + else if(command[1].matches(NNTPConnection.MESSAGE_ID_PATTERN)) + { + // Message-ID + article = Article.getByMessageID(command[1]); + if (article == null) + { + printStatus(430, "no such article found"); + return; + } + } + else + { + // Message Number + try + { + long aid = Long.parseLong(command[1]); + article = Article.getByArticleNumber(aid, getCurrentGroup()); + } + catch(NumberFormatException ex) + { + ex.printStackTrace(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + if (article == null) + { + printStatus(423, "no such article number in this group"); + return; + } + setCurrentArticle(article); + } + + printStatus(223, article.getIndexInGroup(getCurrentGroup()) + " " + article.getMessageID() + + " article retrieved - request text separately"); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/UnsupportedCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/UnsupportedCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,51 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import org.sonews.daemon.NNTPConnection; + +/** + * A default "Unsupported Command". Simply returns error code 500 and a + * "command not supported" message. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class UnsupportedCommand extends AbstractCommand +{ + + public UnsupportedCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException + { + printStatus(500, "command not supported"); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/XDaemonCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/XDaemonCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,237 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.net.InetSocketAddress; +import java.sql.SQLException; +import java.util.List; +import org.sonews.daemon.BootstrapConfig; +import org.sonews.daemon.Config; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.storage.Database; +import org.sonews.daemon.storage.Group; +import org.sonews.feed.FeedManager; +import org.sonews.feed.Subscription; +import org.sonews.util.Stats; + +/** + * The XDAEMON command allows a client to get/set properties of the + * running server daemon. Only locally connected clients are allowed to + * use this command. + * The restriction to localhost connection can be suppressed by overriding + * the sonews.xdaemon.host bootstrap config property. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class XDaemonCommand extends AbstractCommand +{ + + public XDaemonCommand(NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + // TODO: Refactor this method to reduce complexity! + @Override + public void processLine(String line) throws IOException, SQLException + { + InetSocketAddress addr = (InetSocketAddress)connection.getChannel().socket() + .getRemoteSocketAddress(); + if(addr.getHostName().equals( + BootstrapConfig.getInstance().get(BootstrapConfig.XDAEMON_HOST, "localhost"))) + { + String[] commands = line.split(" ", 4); + if(commands.length == 3 && commands[1].equalsIgnoreCase("LIST")) + { + if(commands[2].equalsIgnoreCase("CONFIGKEYS")) + { + printStatus(200, "list of available config keys follows"); + for(String key : Config.AVAILABLE_KEYS) + { + println(key); + } + println("."); + } + else if(commands[2].equalsIgnoreCase("PEERINGRULES")) + { + List pull = + Database.getInstance().getSubscriptions(FeedManager.TYPE_PULL); + List push = + Database.getInstance().getSubscriptions(FeedManager.TYPE_PUSH); + printStatus(200,"list of peering rules follows"); + for(Subscription sub : pull) + { + println("PULL " + sub.getHost() + ":" + sub.getPort() + + " " + sub.getGroup()); + } + for(Subscription sub : push) + { + println("PUSH " + sub.getHost() + ":" + sub.getPort() + + " " + sub.getGroup()); + } + println("."); + } + else + { + printStatus(501, "unknown sub command"); + } + } + else if(commands.length == 3 && commands[1].equalsIgnoreCase("DELETE")) + { + Database.getInstance().delete(commands[2]); + printStatus(200, "article " + commands[2] + " deleted"); + } + else if(commands.length == 4 && commands[1].equalsIgnoreCase("GROUPADD")) + { + Database.getInstance().addGroup(commands[2], Integer.parseInt(commands[3])); + printStatus(200, "group " + commands[2] + " created"); + } + else if(commands.length == 3 && commands[1].equalsIgnoreCase("GROUPDEL")) + { + Group group = Database.getInstance().getGroup(commands[2]); + if(group == null) + { + printStatus(400, "group not found"); + } + else + { + group.setFlag(Group.DELETED); + printStatus(200, "group " + commands[2] + " marked as deleted"); + } + } + else if(commands.length == 4 && commands[1].equalsIgnoreCase("SET")) + { + String key = commands[2]; + String val = commands[3]; + Config.getInstance().set(key, val); + printStatus(200, "new config value set"); + } + else if(commands.length == 3 && commands[1].equalsIgnoreCase("GET")) + { + String key = commands[2]; + String val = Config.getInstance().get(key, null); + if(val != null) + { + printStatus(200, "config value for " + key + " follows"); + println(val); + println("."); + } + else + { + printStatus(400, "config value not set"); + } + } + else if(commands.length >= 3 && commands[1].equalsIgnoreCase("LOG")) + { + Group group = null; + if(commands.length > 3) + { + group = Group.getByName(commands[3]); + } + + if(commands[2].equalsIgnoreCase("CONNECTED_CLIENTS")) + { + printStatus(200, "number of connections follow"); + println(Integer.toString(Stats.getInstance().connectedClients())); + println("."); + } + else if(commands[2].equalsIgnoreCase("POSTED_NEWS")) + { + printStatus(200, "hourly numbers of posted news yesterday"); + for(int n = 0; n < 24; n++) + { + println(n + " " + Stats.getInstance() + .getYesterdaysEvents(Stats.POSTED_NEWS, n, group)); + } + println("."); + } + else if(commands[2].equalsIgnoreCase("GATEWAYED_NEWS")) + { + printStatus(200, "hourly numbers of gatewayed news yesterday"); + for(int n = 0; n < 24; n++) + { + println(n + " " + Stats.getInstance() + .getYesterdaysEvents(Stats.GATEWAYED_NEWS, n, group)); + } + println("."); + } + else if(commands[2].equalsIgnoreCase("TRANSMITTED_NEWS")) + { + printStatus(200, "hourly numbers of news transmitted to peers yesterday"); + for(int n = 0; n < 24; n++) + { + println(n + " " + Stats.getInstance() + .getYesterdaysEvents(Stats.FEEDED_NEWS, n, group)); + } + println("."); + } + else if(commands[2].equalsIgnoreCase("HOSTED_NEWS")) + { + printStatus(200, "number of overall hosted news"); + println(Integer.toString(Stats.getInstance().getNumberOfNews())); + println("."); + } + else if(commands[2].equalsIgnoreCase("HOSTED_GROUPS")) + { + printStatus(200, "number of hosted groups"); + println(Integer.toString(Stats.getInstance().getNumberOfGroups())); + println("."); + } + else if(commands[2].equalsIgnoreCase("POSTED_NEWS_PER_HOUR")) + { + printStatus(200, "posted news per hour"); + println(Double.toString(Stats.getInstance().postedPerHour(-1))); + println("."); + } + else if(commands[2].equalsIgnoreCase("FEEDED_NEWS_PER_HOUR")) + { + printStatus(200, "feeded news per hour"); + println(Double.toString(Stats.getInstance().feededPerHour(-1))); + println("."); + } + else if(commands[2].equalsIgnoreCase("GATEWAYED_NEWS_PER_HOUR")) + { + printStatus(200, "gatewayed news per hour"); + println(Double.toString(Stats.getInstance().gatewayedPerHour(-1))); + println("."); + } + else + { + printStatus(501, "unknown sub command"); + } + } + else + { + printStatus(500, "invalid command usage"); + } + } + else + { + printStatus(500, "not allowed"); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/XPatCommand.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/XPatCommand.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,89 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import java.sql.SQLException; +import org.sonews.daemon.NNTPConnection; + +/** + * 
+ *   XPAT header range| pat [pat...]
+ *
+ *   The XPAT command is used to retrieve specific headers from
+ *   specific articles, based on pattern matching on the contents of
+ *   the header. This command was first available in INN.
+ *
+ *   The required header parameter is the name of a header line (e.g.
+ *   "subject") in a news group article. See RFC-1036 for a list
+ *   of valid header lines. The required range argument may be
+ *   any of the following:
+ *               an article number
+ *               an article number followed by a dash to indicate
+ *                  all following
+ *               an article number followed by a dash followed by
+ *                  another article number
+ *
+ *   The required message-id argument indicates a specific
+ *   article. The range and message-id arguments are mutually
+ *   exclusive. At least one pattern in wildmat must be specified
+ *   as well. If there are additional arguments the are joined
+ *   together separated by a single space to form one complete
+ *   pattern. Successful responses start with a 221 response
+ *   followed by a the headers from all messages in which the
+ *   pattern matched the contents of the specified header line. This
+ *   includes an empty list. Once the output is complete, a period
+ *   is sent on a line by itself. If the optional argument is a
+ *   message-id and no such article exists, the 430 error response
+ *   is returned. A 502 response will be returned if the client only
+ *   has permission to transfer articles.
+ *
+ *   Responses
+ *
+ *       221 Header follows
+ *       430 no such article
+ *       502 no permission
+ *
+ * [Source:"draft-ietf-nntp-imp-02.txt"] [Copyright: 1998 S. Barber] + * + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class XPatCommand extends AbstractCommand +{ + + public XPatCommand(final NNTPConnection conn) + { + super(conn); + } + + @Override + public boolean hasFinished() + { + return true; + } + + @Override + public void processLine(final String line) + throws IOException, SQLException + { + printStatus(500, "not (yet) supported"); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/command/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains a class for every NNTP command. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains basic classes of the daemon. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/storage/Article.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/storage/Article.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,401 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.storage; + +import org.sonews.daemon.Config; +import java.io.BufferedReader; +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.nio.charset.Charset; +import java.sql.SQLException; +import java.util.UUID; +import java.util.ArrayList; +import java.util.Enumeration; +import java.util.List; +import javax.mail.Header; +import javax.mail.Message; +import javax.mail.MessagingException; +import javax.mail.Multipart; +import javax.mail.internet.InternetHeaders; +import javax.mail.internet.MimeUtility; +import org.sonews.util.Log; + +/** + * Represents a newsgroup article. + * @author Christian Lins + * @author Denis Schwerdel + * @since n3tpd/0.1 + */ +public class Article extends ArticleHead +{ + + /** + * Loads the Article identified by the given ID from the Database. + * @param messageID + * @return null if Article is not found or if an error occurred. + */ + public static Article getByMessageID(final String messageID) + { + try + { + return Database.getInstance().getArticle(messageID); + } + catch(SQLException ex) + { + ex.printStackTrace(); + return null; + } + } + + public static Article getByArticleNumber(long articleIndex, Group group) + throws SQLException + { + return Database.getInstance().getArticle(articleIndex, group.getID()); + } + + private String body = ""; + private String headerSrc = null; + + /** + * Default constructor. + */ + public Article() + { + } + + /** + * Creates a new Article object using the date from the given + * raw data. + * This construction has only package visibility. + */ + Article(String headers, String body) + { + try + { + this.body = body; + + // Parse the header + this.headers = new InternetHeaders( + new ByteArrayInputStream(headers.getBytes())); + + this.headerSrc = headers; + } + catch(MessagingException ex) + { + ex.printStackTrace(); + } + } + + /** + * Creates an Article instance using the data from the javax.mail.Message + * object. + * @see javax.mail.Message + * @param msg + * @throws IOException + * @throws MessagingException + */ + public Article(final Message msg) + throws IOException, MessagingException + { + this.headers = new InternetHeaders(); + + for(Enumeration e = msg.getAllHeaders() ; e.hasMoreElements();) + { + final Header header = (Header)e.nextElement(); + this.headers.addHeader(header.getName(), header.getValue()); + } + + // The "content" of the message can be a String if it's a simple text/plain + // message, a Multipart object or an InputStream if the content is unknown. + final Object content = msg.getContent(); + if(content instanceof String) + { + this.body = (String)content; + } + else if(content instanceof Multipart) // probably subclass MimeMultipart + { + // We're are not interested in the different parts of the MultipartMessage, + // so we simply read in all data which *can* be huge. + InputStream in = msg.getInputStream(); + this.body = readContent(in); + } + else if(content instanceof InputStream) + { + // The message format is unknown to the Message class, but we can + // simply read in the whole message data. + this.body = readContent((InputStream)content); + } + else + { + // Unknown content is probably a malformed mail we should skip. + // On the other hand we produce an inconsistent mail mirror, but no + // mail system must transport invalid content. + Log.msg("Skipping message due to unknown content. Throwing exception...", true); + throw new MessagingException("Unknown content: " + content); + } + + // Validate headers + validateHeaders(); + } + + /** + * Reads lines from the given InputString into a String object. + * TODO: Move this generalized method to org.sonews.util.io.Resource. + * @param in + * @return + * @throws IOException + */ + private String readContent(InputStream in) + throws IOException + { + StringBuilder buf = new StringBuilder(); + + BufferedReader rin = new BufferedReader(new InputStreamReader(in)); + String line = rin.readLine(); + while(line != null) + { + buf.append('\n'); + buf.append(line); + line = rin.readLine(); + } + + return buf.toString(); + } + + /** + * Removes the header identified by the given key. + * @param headerKey + */ + public void removeHeader(final String headerKey) + { + this.headers.removeHeader(headerKey); + this.headerSrc = null; + } + + /** + * Generates a message id for this article and sets it into + * the header object. You have to update the Database manually to make this + * change persistent. + * Note: a Message-ID should never be changed and only generated once. + */ + private String generateMessageID() + { + String msgID = "<" + UUID.randomUUID() + "@" + + Config.getInstance().get(Config.HOSTNAME, "localhost") + ">"; + + this.headers.setHeader(Headers.MESSAGE_ID, msgID); + + return msgID; + } + + /** + * Returns the body string. + */ + public String getBody() + { + return body; + } + + /** + * @return Charset of the body text + */ + public Charset getBodyCharset() + { + // We espect something like + // Content-Type: text/plain; charset=ISO-8859-15 + String contentType = getHeader(Headers.CONTENT_TYPE)[0]; + int idxCharsetStart = contentType.indexOf("charset=") + "charset=".length(); + int idxCharsetEnd = contentType.indexOf(";", idxCharsetStart); + + String charsetName = "UTF-8"; + if(idxCharsetStart >= 0 && idxCharsetStart < contentType.length()) + { + if(idxCharsetEnd < 0) + { + charsetName = contentType.substring(idxCharsetStart); + } + else + { + charsetName = contentType.substring(idxCharsetStart, idxCharsetEnd); + } + } + + // Sometimes there are '"' around the name + if(charsetName.length() > 2 && + charsetName.charAt(0) == '"' && charsetName.endsWith("\"")) + { + charsetName = charsetName.substring(1, charsetName.length() - 2); + } + + // Create charset + Charset charset = Charset.forName("UTF-8"); // This MUST be supported by JVM + try + { + charset = Charset.forName(charsetName); + } + catch(Exception ex) + { + Log.msg(ex.getMessage(), false); + Log.msg("Article.getBodyCharset(): Unknown charset: " + charsetName, false); + } + return charset; + } + + /** + * @return Numerical IDs of the newsgroups this Article belongs to. + */ + List getGroups() + { + String[] groupnames = getHeader(Headers.NEWSGROUPS)[0].split(","); + ArrayList groups = new ArrayList(); + + try + { + for(String newsgroup : groupnames) + { + newsgroup = newsgroup.trim(); + Group group = Database.getInstance().getGroup(newsgroup); + if(group != null && // If the server does not provide the group, ignore it + !groups.contains(group)) // Yes, there may be duplicates + { + groups.add(group); + } + } + } + catch (SQLException ex) + { + ex.printStackTrace(); + return null; + } + return groups; + } + + public void setBody(String body) + { + this.body = body; + } + + /** + * + * @param groupname Name(s) of newsgroups + */ + public void setGroup(String groupname) + { + this.headers.setHeader(Headers.NEWSGROUPS, groupname); + } + + public String getMessageID() + { + String[] msgID = getHeader(Headers.MESSAGE_ID); + return msgID[0]; + } + + public Enumeration getAllHeaders() + { + return this.headers.getAllHeaders(); + } + + /** + * @return Header source code of this Article. + */ + public String getHeaderSource() + { + if(this.headerSrc != null) + { + return this.headerSrc; + } + + StringBuffer buf = new StringBuffer(); + + for(Enumeration en = this.headers.getAllHeaders(); en.hasMoreElements();) + { + Header entry = (Header)en.nextElement(); + + buf.append(entry.getName()); + buf.append(": "); + buf.append( + MimeUtility.fold(entry.getName().length() + 2, entry.getValue())); + + if(en.hasMoreElements()) + { + buf.append("\r\n"); + } + } + + this.headerSrc = buf.toString(); + return this.headerSrc; + } + + public long getIndexInGroup(Group group) + throws SQLException + { + return Database.getInstance().getArticleIndex(this, group); + } + + /** + * Sets the headers of this Article. If headers contain no + * Message-Id a new one is created. + * @param headers + */ + public void setHeaders(InternetHeaders headers) + { + this.headers = headers; + validateHeaders(); + } + + /** + * @return String containing the Message-ID. + */ + @Override + public String toString() + { + return getMessageID(); + } + + /** + * Checks some headers for their validity and generates an + * appropriate Path-header for this host if not yet existing. + * This method is called by some Article constructors and the + * method setHeaders(). + * @return true if something on the headers was changed. + */ + private void validateHeaders() + { + // Check for valid Path-header + final String path = getHeader(Headers.PATH)[0]; + final String host = Config.getInstance().get(Config.HOSTNAME, "localhost"); + if(!path.startsWith(host)) + { + StringBuffer pathBuf = new StringBuffer(); + pathBuf.append(host); + pathBuf.append('!'); + pathBuf.append(path); + this.headers.setHeader(Headers.PATH, pathBuf.toString()); + } + + // Generate a messageID if no one is existing + if(getMessageID().equals("")) + { + generateMessageID(); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/storage/ArticleHead.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/storage/ArticleHead.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,78 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.storage; + +import java.io.ByteArrayInputStream; +import javax.mail.MessagingException; +import javax.mail.internet.InternetHeaders; + +/** + * An article with no body only headers. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ArticleHead +{ + + protected InternetHeaders headers; + + protected ArticleHead() + { + } + + public ArticleHead(String headers) + { + try + { + // Parse the header + this.headers = new InternetHeaders( + new ByteArrayInputStream(headers.getBytes())); + } + catch(MessagingException ex) + { + ex.printStackTrace(); + } + } + + /** + * Returns the header field with given name. + * @param name + * @return Header values or empty string. + */ + public String[] getHeader(String name) + { + String[] ret = this.headers.getHeader(name); + if(ret == null) + { + ret = new String[]{""}; + } + return ret; + } + + /** + * Sets the header value identified through the header name. + * @param name + * @param value + */ + public void setHeader(String name, String value) + { + this.headers.setHeader(name, value); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/storage/Database.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/storage/Database.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1353 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.storage; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.Statement; +import java.sql.PreparedStatement; +import java.util.ArrayList; +import java.util.Enumeration; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import javax.mail.Header; +import javax.mail.internet.InternetAddress; +import javax.mail.internet.MimeUtility; +import org.sonews.daemon.BootstrapConfig; +import org.sonews.util.Log; +import org.sonews.feed.Subscription; +import org.sonews.util.Pair; + +/** + * Database facade class. + * @author Christian Lins + * @since sonews/0.5.0 + */ +// TODO: Refactor this class to reduce size (e.g. ArticleDatabase GroupDatabase) +public class Database +{ + + public static final int MAX_RESTARTS = 3; + + private static final Map instances + = new ConcurrentHashMap(); + + /** + * @return Instance of the current Database backend. Returns null if an error + * has occurred. + */ + public static Database getInstance(boolean create) + throws SQLException + { + if(!instances.containsKey(Thread.currentThread()) && create) + { + Database db = new Database(); + db.arise(); + instances.put(Thread.currentThread(), db); + return db; + } + else + { + return instances.get(Thread.currentThread()); + } + } + + public static Database getInstance() + throws SQLException + { + return getInstance(true); + } + + private Connection conn = null; + private PreparedStatement pstmtAddArticle1 = null; + private PreparedStatement pstmtAddArticle2 = null; + private PreparedStatement pstmtAddArticle3 = null; + private PreparedStatement pstmtAddArticle4 = null; + private PreparedStatement pstmtAddGroup0 = null; + private PreparedStatement pstmtAddEvent = null; + private PreparedStatement pstmtCountArticles = null; + private PreparedStatement pstmtCountGroups = null; + private PreparedStatement pstmtDeleteArticle0 = null; + private PreparedStatement pstmtGetArticle0 = null; + private PreparedStatement pstmtGetArticle1 = null; + private PreparedStatement pstmtGetArticleHeaders = null; + private PreparedStatement pstmtGetArticleHeads = null; + private PreparedStatement pstmtGetArticleIDs = null; + private PreparedStatement pstmtGetArticleIndex = null; + private PreparedStatement pstmtGetConfigValue = null; + private PreparedStatement pstmtGetEventsCount0 = null; + private PreparedStatement pstmtGetEventsCount1 = null; + private PreparedStatement pstmtGetGroupForList = null; + private PreparedStatement pstmtGetGroup0 = null; + private PreparedStatement pstmtGetGroup1 = null; + private PreparedStatement pstmtGetFirstArticleNumber = null; + private PreparedStatement pstmtGetListForGroup = null; + private PreparedStatement pstmtGetLastArticleNumber = null; + private PreparedStatement pstmtGetMaxArticleID = null; + private PreparedStatement pstmtGetMaxArticleIndex = null; + private PreparedStatement pstmtGetPostingsCount = null; + private PreparedStatement pstmtGetSubscriptions = null; + private PreparedStatement pstmtIsArticleExisting = null; + private PreparedStatement pstmtIsGroupExisting = null; + private PreparedStatement pstmtSetConfigValue0 = null; + private PreparedStatement pstmtSetConfigValue1 = null; + + /** How many times the database connection was reinitialized */ + private int restarts = 0; + + /** + * Rises the database: reconnect and recreate all prepared statements. + * @throws java.lang.SQLException + */ + private void arise() + throws SQLException + { + try + { + // Load database driver + Class.forName( + BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_DBMSDRIVER, "java.lang.Object")); + + // Establish database connection + this.conn = DriverManager.getConnection( + BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_DATABASE, ""), + BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_USER, "root"), + BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_PASSWORD, "")); + + this.conn.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE); + if(this.conn.getTransactionIsolation() != Connection.TRANSACTION_SERIALIZABLE) + { + Log.msg("Warning: Database is NOT fully serializable!", false); + } + + // Prepare statements for method addArticle() + this.pstmtAddArticle1 = conn.prepareStatement( + "INSERT INTO articles (article_id, body) VALUES(?, ?)"); + this.pstmtAddArticle2 = conn.prepareStatement( + "INSERT INTO headers (article_id, header_key, header_value, header_index) " + + "VALUES (?, ?, ?, ?)"); + this.pstmtAddArticle3 = conn.prepareStatement( + "INSERT INTO postings (group_id, article_id, article_index)" + + "VALUES (?, ?, ?)"); + this.pstmtAddArticle4 = conn.prepareStatement( + "INSERT INTO article_ids (article_id, message_id) VALUES (?, ?)"); + + // Prepare statement for method addStatValue() + this.pstmtAddEvent = conn.prepareStatement( + "INSERT INTO events VALUES (?, ?, ?)"); + + // Prepare statement for method addGroup() + this.pstmtAddGroup0 = conn.prepareStatement( + "INSERT INTO groups (name, flags) VALUES (?, ?)"); + + // Prepare statement for method countArticles() + this.pstmtCountArticles = conn.prepareStatement( + "SELECT Count(article_id) FROM article_ids"); + + // Prepare statement for method countGroups() + this.pstmtCountGroups = conn.prepareStatement( + "SELECT Count(group_id) FROM groups WHERE " + + "flags & " + Group.DELETED + " = 0"); + + // Prepare statements for method delete(article) + this.pstmtDeleteArticle0 = conn.prepareStatement( + "DELETE FROM articles WHERE article_id = " + + "(SELECT article_id FROM article_ids WHERE message_id = ?)"); + + // Prepare statements for methods getArticle() + this.pstmtGetArticle0 = conn.prepareStatement( + "SELECT * FROM articles WHERE article_id = " + + "(SELECT article_id FROM article_ids WHERE message_id = ?)"); + this.pstmtGetArticle1 = conn.prepareStatement( + "SELECT * FROM articles WHERE article_id = " + + "(SELECT article_id FROM postings WHERE " + + "article_index = ? AND group_id = ?)"); + + // Prepare statement for method getArticleHeaders() + this.pstmtGetArticleHeaders = conn.prepareStatement( + "SELECT header_key, header_value FROM headers WHERE article_id = ? " + + "ORDER BY header_index ASC"); + + this.pstmtGetArticleIDs = conn.prepareStatement( + "SELECT article_index FROM postings WHERE group_id = ?"); + + // Prepare statement for method getArticleIndex + this.pstmtGetArticleIndex = conn.prepareStatement( + "SELECT article_index FROM postings WHERE " + + "article_id = (SELECT article_id FROM article_ids " + + "WHERE message_id = ?) " + + " AND group_id = ?"); + + // Prepare statements for method getArticleHeads() + this.pstmtGetArticleHeads = conn.prepareStatement( + "SELECT article_id, article_index FROM postings WHERE " + + "postings.group_id = ? AND article_index >= ? AND " + + "article_index <= ?"); + + // Prepare statements for method getConfigValue() + this.pstmtGetConfigValue = conn.prepareStatement( + "SELECT config_value FROM config WHERE config_key = ?"); + + // Prepare statements for method getEventsCount() + this.pstmtGetEventsCount0 = conn.prepareStatement( + "SELECT Count(*) FROM events WHERE event_key = ? AND " + + "event_time >= ? AND event_time < ?"); + + this.pstmtGetEventsCount1 = conn.prepareStatement( + "SELECT Count(*) FROM events WHERE event_key = ? AND " + + "event_time >= ? AND event_time < ? AND group_id = ?"); + + // Prepare statement for method getGroupForList() + this.pstmtGetGroupForList = conn.prepareStatement( + "SELECT name FROM groups INNER JOIN groups2list " + + "ON groups.group_id = groups2list.group_id " + + "WHERE groups2list.listaddress = ?"); + + // Prepare statement for method getGroup() + this.pstmtGetGroup0 = conn.prepareStatement( + "SELECT group_id, flags FROM groups WHERE Name = ?"); + this.pstmtGetGroup1 = conn.prepareStatement( + "SELECT name FROM groups WHERE group_id = ?"); + + // Prepare statement for method getLastArticleNumber() + this.pstmtGetLastArticleNumber = conn.prepareStatement( + "SELECT Max(article_index) FROM postings WHERE group_id = ?"); + + // Prepare statement for method getListForGroup() + this.pstmtGetListForGroup = conn.prepareStatement( + "SELECT listaddress FROM groups2list INNER JOIN groups " + + "ON groups.group_id = groups2list.group_id WHERE name = ?"); + + // Prepare statement for method getMaxArticleID() + this.pstmtGetMaxArticleID = conn.prepareStatement( + "SELECT Max(article_id) FROM articles"); + + // Prepare statement for method getMaxArticleIndex() + this.pstmtGetMaxArticleIndex = conn.prepareStatement( + "SELECT Max(article_index) FROM postings WHERE group_id = ?"); + + // Prepare statement for method getFirstArticleNumber() + this.pstmtGetFirstArticleNumber = conn.prepareStatement( + "SELECT Min(article_index) FROM postings WHERE group_id = ?"); + + // Prepare statement for method getPostingsCount() + this.pstmtGetPostingsCount = conn.prepareStatement( + "SELECT Count(*) FROM postings NATURAL JOIN groups " + + "WHERE groups.name = ?"); + + // Prepare statement for method getSubscriptions() + this.pstmtGetSubscriptions = conn.prepareStatement( + "SELECT host, port, name FROM peers NATURAL JOIN " + + "peer_subscriptions NATURAL JOIN groups WHERE feedtype = ?"); + + // Prepare statement for method isArticleExisting() + this.pstmtIsArticleExisting = conn.prepareStatement( + "SELECT Count(article_id) FROM article_ids WHERE message_id = ?"); + + // Prepare statement for method isGroupExisting() + this.pstmtIsGroupExisting = conn.prepareStatement( + "SELECT * FROM groups WHERE name = ?"); + + // Prepare statement for method setConfigValue() + this.pstmtSetConfigValue0 = conn.prepareStatement( + "DELETE FROM config WHERE config_key = ?"); + this.pstmtSetConfigValue1 = conn.prepareStatement( + "INSERT INTO config VALUES(?, ?)"); + } + catch(ClassNotFoundException ex) + { + throw new Error("JDBC Driver not found!", ex); + } + } + + /** + * Adds an article to the database. + * @param article + * @return + * @throws java.sql.SQLException + */ + public void addArticle(final Article article) + throws SQLException + { + try + { + this.conn.setAutoCommit(false); + + int newArticleID = getMaxArticleID() + 1; + + // Fill prepared statement with values; + // writes body to article table + pstmtAddArticle1.setInt(1, newArticleID); + pstmtAddArticle1.setBytes(2, article.getBody().getBytes()); + pstmtAddArticle1.execute(); + + // Add headers + Enumeration headers = article.getAllHeaders(); + for(int n = 0; headers.hasMoreElements(); n++) + { + Header header = (Header)headers.nextElement(); + pstmtAddArticle2.setInt(1, newArticleID); + pstmtAddArticle2.setString(2, header.getName().toLowerCase()); + pstmtAddArticle2.setString(3, + header.getValue().replaceAll("[\r\n]", "")); + pstmtAddArticle2.setInt(4, n); + pstmtAddArticle2.execute(); + } + + // For each newsgroup add a reference + List groups = article.getGroups(); + for(Group group : groups) + { + pstmtAddArticle3.setLong(1, group.getID()); + pstmtAddArticle3.setInt(2, newArticleID); + pstmtAddArticle3.setLong(3, getMaxArticleIndex(group.getID()) + 1); + pstmtAddArticle3.execute(); + } + + // Write message-id to article_ids table + this.pstmtAddArticle4.setInt(1, newArticleID); + this.pstmtAddArticle4.setString(2, article.getMessageID()); + this.pstmtAddArticle4.execute(); + + this.conn.commit(); + this.conn.setAutoCommit(true); + + this.restarts = 0; // Reset error count + } + catch(SQLException ex) + { + try + { + this.conn.rollback(); // Rollback changes + } + catch(SQLException ex2) + { + Log.msg("Rollback of addArticle() failed: " + ex2, false); + } + + try + { + this.conn.setAutoCommit(true); // and release locks + } + catch(SQLException ex2) + { + Log.msg("setAutoCommit(true) of addArticle() failed: " + ex2, false); + } + + restartConnection(ex); + addArticle(article); + } + } + + /** + * Adds a group to the Database. This method is not accessible via NNTP. + * @param name + * @throws java.sql.SQLException + */ + public void addGroup(String name, int flags) + throws SQLException + { + try + { + this.conn.setAutoCommit(false); + pstmtAddGroup0.setString(1, name); + pstmtAddGroup0.setInt(2, flags); + + pstmtAddGroup0.executeUpdate(); + this.conn.commit(); + this.conn.setAutoCommit(true); + this.restarts = 0; // Reset error count + } + catch(SQLException ex) + { + this.conn.rollback(); + this.conn.setAutoCommit(true); + restartConnection(ex); + addGroup(name, flags); + } + } + + public void addEvent(long time, byte type, long gid) + throws SQLException + { + try + { + this.conn.setAutoCommit(false); + this.pstmtAddEvent.setLong(1, time); + this.pstmtAddEvent.setInt(2, type); + this.pstmtAddEvent.setLong(3, gid); + this.pstmtAddEvent.executeUpdate(); + this.conn.commit(); + this.conn.setAutoCommit(true); + this.restarts = 0; + } + catch(SQLException ex) + { + this.conn.rollback(); + this.conn.setAutoCommit(true); + + restartConnection(ex); + addEvent(time, type, gid); + } + } + + public int countArticles() + throws SQLException + { + ResultSet rs = null; + + try + { + rs = this.pstmtCountArticles.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return countArticles(); + } + finally + { + if(rs != null) + { + rs.close(); + restarts = 0; + } + } + } + + public int countGroups() + throws SQLException + { + ResultSet rs = null; + + try + { + rs = this.pstmtCountGroups.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return countGroups(); + } + finally + { + if(rs != null) + { + rs.close(); + restarts = 0; + } + } + } + + public void delete(final String messageID) + throws SQLException + { + try + { + this.conn.setAutoCommit(false); + + this.pstmtDeleteArticle0.setString(1, messageID); + ResultSet rs = this.pstmtDeleteArticle0.executeQuery(); + rs.next(); + + // We trust the ON DELETE CASCADE functionality to delete + // orphaned references + + this.conn.commit(); + this.conn.setAutoCommit(true); + } + catch(SQLException ex) + { + throw ex; + } + } + + public Article getArticle(String messageID) + throws SQLException + { + ResultSet rs = null; + try + { + pstmtGetArticle0.setString(1, messageID); + rs = pstmtGetArticle0.executeQuery(); + + if(!rs.next()) + { + return null; + } + else + { + String body = new String(rs.getBytes("body")); + String headers = getArticleHeaders(rs.getInt("article_id")); + return new Article(headers, body); + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticle(messageID); + } + finally + { + if(rs != null) + { + rs.close(); + restarts = 0; // Reset error count + } + } + } + + /** + * Retrieves an article by its ID. + * @param articleID + * @return + * @throws java.sql.SQLException + */ + public Article getArticle(long articleIndex, long gid) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticle1.setLong(1, articleIndex); + this.pstmtGetArticle1.setLong(2, gid); + + rs = this.pstmtGetArticle1.executeQuery(); + + if(rs.next()) + { + String body = new String(rs.getBytes("body")); + String headers = getArticleHeaders(rs.getInt("article_id")); + return new Article(headers, body); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticle(articleIndex, gid); + } + finally + { + if(rs != null) + { + rs.close(); + restarts = 0; + } + } + } + + public String getArticleHeaders(long articleID) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticleHeaders.setLong(1, articleID); + rs = this.pstmtGetArticleHeaders.executeQuery(); + + StringBuilder buf = new StringBuilder(); + if(rs.next()) + { + for(;;) + { + buf.append(rs.getString(1)); // key + buf.append(": "); + String foldedValue = MimeUtility.fold(0, rs.getString(2)); + buf.append(foldedValue); // value + if(rs.next()) + { + buf.append("\r\n"); + } + else + { + break; + } + } + } + + return buf.toString(); + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleHeaders(articleID); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public long getArticleIndex(Article article, Group group) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticleIndex.setString(1, article.getMessageID()); + this.pstmtGetArticleIndex.setLong(2, group.getID()); + + rs = this.pstmtGetArticleIndex.executeQuery(); + if(rs.next()) + { + return rs.getLong(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleIndex(article, group); + } + finally + { + if(rs != null) + rs.close(); + } + } + + /** + * Returns a list of Long/Article Pairs. + * @throws java.sql.SQLException + */ + public List> getArticleHeads(Group group, int first, int last) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticleHeads.setLong(1, group.getID()); + this.pstmtGetArticleHeads.setInt(2, first); + this.pstmtGetArticleHeads.setInt(3, last); + rs = pstmtGetArticleHeads.executeQuery(); + + List> articles + = new ArrayList>(); + + while (rs.next()) + { + long aid = rs.getLong("article_id"); + long aidx = rs.getLong("article_index"); + String headers = getArticleHeaders(aid); + articles.add(new Pair(aidx, + new ArticleHead(headers))); + } + + return articles; + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleHeads(group, first, last); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public List getArticleNumbers(long gid) + throws SQLException + { + ResultSet rs = null; + try + { + List ids = new ArrayList(); + this.pstmtGetArticleIDs.setLong(1, gid); + rs = this.pstmtGetArticleIDs.executeQuery(); + while(rs.next()) + { + ids.add(rs.getLong(1)); + } + return ids; + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleNumbers(gid); + } + finally + { + if(rs != null) + { + rs.close(); + restarts = 0; // Clear the restart count after successful request + } + } + } + + public String getConfigValue(String key) + throws SQLException + { + ResultSet rs = null; + try + { + this.pstmtGetConfigValue.setString(1, key); + + rs = this.pstmtGetConfigValue.executeQuery(); + if(rs.next()) + { + return rs.getString(1); // First data on index 1 not 0 + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getConfigValue(key); + } + finally + { + if(rs != null) + { + rs.close(); + restarts = 0; // Clear the restart count after successful request + } + } + } + + public int getEventsCount(byte type, long start, long end, Group group) + throws SQLException + { + ResultSet rs = null; + + try + { + if(group == null) + { + this.pstmtGetEventsCount0.setInt(1, type); + this.pstmtGetEventsCount0.setLong(2, start); + this.pstmtGetEventsCount0.setLong(3, end); + rs = this.pstmtGetEventsCount0.executeQuery(); + } + else + { + this.pstmtGetEventsCount1.setInt(1, type); + this.pstmtGetEventsCount1.setLong(2, start); + this.pstmtGetEventsCount1.setLong(3, end); + this.pstmtGetEventsCount1.setLong(4, group.getID()); + rs = this.pstmtGetEventsCount1.executeQuery(); + } + + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getEventsCount(type, start, end, group); + } + finally + { + if(rs != null) + rs.close(); + } + } + + /** + * Reads all Groups from the Database. + * @return + * @throws java.sql.SQLException + */ + public List getGroups() + throws SQLException + { + ResultSet rs; + List buffer = new ArrayList(); + Statement stmt = null; + + try + { + stmt = conn.createStatement(); + rs = stmt.executeQuery("SELECT * FROM groups ORDER BY name"); + + while(rs.next()) + { + String name = rs.getString("name"); + long id = rs.getLong("group_id"); + int flags = rs.getInt("flags"); + + Group group = new Group(name, id, flags); + buffer.add(group); + } + + return buffer; + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroups(); + } + finally + { + if(stmt != null) + stmt.close(); // Implicitely closes ResultSets + } + } + + public String getGroupForList(InternetAddress listAddress) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetGroupForList.setString(1, listAddress.getAddress()); + + rs = this.pstmtGetGroupForList.executeQuery(); + if (rs.next()) + { + return rs.getString(1); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroupForList(listAddress); + } + finally + { + if(rs != null) + rs.close(); + } + } + + /** + * Returns the Group that is identified by the name. + * @param name + * @return + * @throws java.sql.SQLException + */ + public Group getGroup(String name) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetGroup0.setString(1, name); + rs = this.pstmtGetGroup0.executeQuery(); + + if (!rs.next()) + { + return null; + } + else + { + long id = rs.getLong("group_id"); + int flags = rs.getInt("flags"); + return new Group(name, id, flags); + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroup(name); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public String getListForGroup(String group) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetListForGroup.setString(1, group); + rs = this.pstmtGetListForGroup.executeQuery(); + if (rs.next()) + { + return rs.getString(1); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getListForGroup(group); + } + finally + { + if(rs != null) + rs.close(); + } + } + + private int getMaxArticleIndex(long groupID) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetMaxArticleIndex.setLong(1, groupID); + rs = this.pstmtGetMaxArticleIndex.executeQuery(); + + int maxIndex = 0; + if (rs.next()) + { + maxIndex = rs.getInt(1); + } + + return maxIndex; + } + catch(SQLException ex) + { + restartConnection(ex); + return getMaxArticleIndex(groupID); + } + finally + { + if(rs != null) + rs.close(); + } + } + + private int getMaxArticleID() + throws SQLException + { + ResultSet rs = null; + + try + { + rs = this.pstmtGetMaxArticleID.executeQuery(); + + int maxIndex = 0; + if (rs.next()) + { + maxIndex = rs.getInt(1); + } + + return maxIndex; + } + catch(SQLException ex) + { + restartConnection(ex); + return getMaxArticleID(); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public int getLastArticleNumber(Group group) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetLastArticleNumber.setLong(1, group.getID()); + rs = this.pstmtGetLastArticleNumber.executeQuery(); + if (rs.next()) + { + return rs.getInt(1); + } + else + { + return 0; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getLastArticleNumber(group); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public int getFirstArticleNumber(Group group) + throws SQLException + { + ResultSet rs = null; + try + { + this.pstmtGetFirstArticleNumber.setLong(1, group.getID()); + rs = this.pstmtGetFirstArticleNumber.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return 0; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getFirstArticleNumber(group); + } + finally + { + if(rs != null) + rs.close(); + } + } + + /** + * Returns a group name identified by the given id. + * @param id + * @return + * @throws java.sql.SQLException + */ + public String getGroup(int id) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetGroup1.setInt(1, id); + rs = this.pstmtGetGroup1.executeQuery(); + + if (rs.next()) + { + return rs.getString(1); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroup(id); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public double getNumberOfEventsPerHour(int key, long gid) + throws SQLException + { + String gidquery = ""; + if(gid >= 0) + { + gidquery = " AND group_id = " + gid; + } + + Statement stmt = null; + ResultSet rs = null; + + try + { + stmt = this.conn.createStatement(); + rs = stmt.executeQuery("SELECT Count(*) / (Max(event_time) - Min(event_time))" + + " * 1000 * 60 * 60 FROM events WHERE event_key = " + key + gidquery); + + if(rs.next()) + { + restarts = 0; // reset error count + return rs.getDouble(1); + } + else + { + return Double.NaN; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getNumberOfEventsPerHour(key, gid); + } + finally + { + if(stmt != null) + { + stmt.close(); + } + + if(rs != null) + { + rs.close(); + } + } + } + + public int getPostingsCount(String groupname) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtGetPostingsCount.setString(1, groupname); + rs = this.pstmtGetPostingsCount.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + Log.msg("Warning: Count on postings return nothing!", true); + return 0; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getPostingsCount(groupname); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public List getSubscriptions(int feedtype) + throws SQLException + { + ResultSet rs = null; + + try + { + List subs = new ArrayList(); + this.pstmtGetSubscriptions.setInt(1, feedtype); + rs = this.pstmtGetSubscriptions.executeQuery(); + + while(rs.next()) + { + String host = rs.getString("host"); + String group = rs.getString("name"); + int port = rs.getInt("port"); + subs.add(new Subscription(host, port, feedtype, group)); + } + + return subs; + } + catch(SQLException ex) + { + restartConnection(ex); + return getSubscriptions(feedtype); + } + finally + { + if(rs != null) + rs.close(); + } + } + + /** + * Checks if there is an article with the given messageid in the Database. + * @param name + * @return + * @throws java.sql.SQLException + */ + public boolean isArticleExisting(String messageID) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtIsArticleExisting.setString(1, messageID); + rs = this.pstmtIsArticleExisting.executeQuery(); + return rs.next() && rs.getInt(1) == 1; + } + catch(SQLException ex) + { + restartConnection(ex); + return isArticleExisting(messageID); + } + finally + { + if(rs != null) + rs.close(); + } + } + + /** + * Checks if there is a group with the given name in the Database. + * @param name + * @return + * @throws java.sql.SQLException + */ + public boolean isGroupExisting(String name) + throws SQLException + { + ResultSet rs = null; + + try + { + this.pstmtIsGroupExisting.setString(1, name); + rs = this.pstmtIsGroupExisting.executeQuery(); + return rs.next(); + } + catch(SQLException ex) + { + restartConnection(ex); + return isGroupExisting(name); + } + finally + { + if(rs != null) + rs.close(); + } + } + + public void setConfigValue(String key, String value) + throws SQLException + { + try + { + conn.setAutoCommit(false); + this.pstmtSetConfigValue0.setString(1, key); + this.pstmtSetConfigValue0.execute(); + this.pstmtSetConfigValue1.setString(1, key); + this.pstmtSetConfigValue1.setString(2, value); + this.pstmtSetConfigValue1.execute(); + conn.commit(); + conn.setAutoCommit(true); + } + catch(SQLException ex) + { + restartConnection(ex); + setConfigValue(key, value); + } + } + + /** + * Closes the Database connection. + */ + public void shutdown() + throws SQLException + { + if(this.conn != null) + { + this.conn.close(); + } + } + + private void restartConnection(SQLException cause) + throws SQLException + { + restarts++; + Log.msg(Thread.currentThread() + + ": Database connection was closed (restart " + restarts + ").", false); + + if(restarts >= MAX_RESTARTS) + { + // Delete the current, probably broken Database instance. + // So no one can use the instance any more. + Database.instances.remove(Thread.currentThread()); + + // Throw the exception upwards + throw cause; + } + + try + { + Thread.sleep(1500L * restarts); + } + catch(InterruptedException ex) + { + Log.msg("Interrupted: " + ex.getMessage(), false); + } + + // Try to properly close the old database connection + try + { + if(this.conn != null) + { + this.conn.close(); + } + } + catch(SQLException ex) + { + Log.msg(ex.getMessage(), true); + } + + try + { + // Try to reinitialize database connection + arise(); + } + catch(SQLException ex) + { + Log.msg(ex.getMessage(), true); + restartConnection(ex); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/storage/Group.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/storage/Group.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,186 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.storage; + +import java.sql.SQLException; +import java.util.List; +import org.sonews.util.Log; +import org.sonews.util.Pair; + +/** + * Represents a logical Group within this newsserver. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class Group +{ + + /** + * If this flag is set the Group is no real newsgroup but a mailing list + * mirror. In that case every posting and receiving mails must go through + * the mailing list gateway. + */ + public static final int MAILINGLIST = 0x1; + + /** + * If this flag is set the Group is marked as readonly and the posting + * is prohibited. This can be useful for groups that are synced only in + * one direction. + */ + public static final int READONLY = 0x2; + + /** + * If this flag is set the Group is marked as deleted and must not occur + * in any output. The deletion is done lazily by a low priority daemon. + */ + public static final int DELETED = 0x128; + + private long id = 0; + private int flags = -1; + private String name = null; + + /** + * Returns a Group identified by its full name. + * @param name + * @return + */ + public static Group getByName(final String name) + { + try + { + return Database.getInstance().getGroup(name); + } + catch(SQLException ex) + { + ex.printStackTrace(); + return null; + } + } + + /** + * @return List of all groups this server handles. + */ + public static List getAll() + { + try + { + return Database.getInstance().getGroups(); + } + catch(SQLException ex) + { + Log.msg(ex.getMessage(), false); + return null; + } + } + + /** + * Private constructor. + * @param name + * @param id + */ + Group(final String name, final long id, final int flags) + { + this.id = id; + this.flags = flags; + this.name = name; + } + + @Override + public boolean equals(Object obj) + { + if(obj instanceof Group) + { + return ((Group)obj).id == this.id; + } + else + { + return false; + } + } + + public List> getArticleHeads(final int first, final int last) + throws SQLException + { + return Database.getInstance().getArticleHeads(this, first, last); + } + + public List getArticleNumbers() + throws SQLException + { + return Database.getInstance().getArticleNumbers(id); + } + + public int getFirstArticleNumber() + throws SQLException + { + return Database.getInstance().getFirstArticleNumber(this); + } + + /** + * Returns the group id. + */ + public long getID() + { + assert id > 0; + + return id; + } + + public boolean isMailingList() + { + return (this.flags & MAILINGLIST) != 0; + } + + public int getLastArticleNumber() + throws SQLException + { + return Database.getInstance().getLastArticleNumber(this); + } + + public String getName() + { + return name; + } + + /** + * Performs this.flags |= flag to set a specified flag and updates the data + * in the Database. + * @param flag + */ + public void setFlag(final int flag) + { + this.flags |= flag; + } + + public void setName(final String name) + { + this.name = name; + } + + /** + * @return Number of posted articles in this group. + * @throws java.sql.SQLException + */ + public int getPostingsCount() + throws SQLException + { + return Database.getInstance().getPostingsCount(this.name); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/storage/Headers.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/storage/Headers.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,51 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.storage; + +/** + * Contains header constants. These header keys are no way complete but all + * headers that are relevant for sonews. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Headers +{ + + public static final String BYTES = "bytes"; + public static final String CONTENT_TYPE = "content-type"; + public static final String CONTROL = "control"; + public static final String DATE = "date"; + public static final String FROM = "from"; + public static final String LINES = "lines"; + public static final String MESSAGE_ID = "message-id"; + public static final String NEWSGROUPS = "newsgroups"; + public static final String NNTP_POSTING_DATE = "nntp-posting-date"; + public static final String NNTP_POSTING_HOST = "nntp-posting-host"; + public static final String PATH = "path"; + public static final String REFERENCES = "references"; + public static final String SUBJECT = "subject"; + public static final String SUPERSEDES = "subersedes"; + public static final String X_COMPLAINTS_TO = "x-complaints-to"; + public static final String X_TRACE = "x-trace"; + public static final String XREF = "xref"; + + private Headers() + {} + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/daemon/storage/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/storage/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,2 @@ +Contains classes of the storage backend and the Group and Article +abstraction. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/feed/AbstractFeeder.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/feed/AbstractFeeder.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,41 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.feed; + +import java.util.ArrayList; +import java.util.List; +import org.sonews.daemon.AbstractDaemon; + +/** + * Base class for PullFeeder and PushFeeder. + * @author Christian Lins + * @since sonews/0.5.0 + */ +abstract class AbstractFeeder extends AbstractDaemon +{ + + /** List of subscriptions that is processed by this feeder */ + protected List subscriptions = new ArrayList(); + + public void addSubscription(final Subscription sub) + { + this.subscriptions.add(sub); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/feed/FeedManager.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/feed/FeedManager.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,71 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.feed; + +import java.sql.SQLException; +import java.util.List; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.Database; + +/** + * Controlls push and pull feeder. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class FeedManager +{ + + public static final int TYPE_PULL = 0; + public static final int TYPE_PUSH = 1; + + private static PullFeeder pullFeeder = new PullFeeder(); + private static PushFeeder pushFeeder = new PushFeeder(); + + /** + * Reads the peer subscriptions from database and starts the appropriate + * PullFeeder or PushFeeder. + */ + public static synchronized void startFeeding() + throws SQLException + { + List subsPull = Database.getInstance() + .getSubscriptions(TYPE_PULL); + for(Subscription sub : subsPull) + { + pullFeeder.addSubscription(sub); + } + pullFeeder.start(); + + List subsPush = Database.getInstance() + .getSubscriptions(TYPE_PUSH); + for(Subscription sub : subsPush) + { + pushFeeder.addSubscription(sub); + } + pushFeeder.start(); + } + + public static void queueForPush(Article article) + { + pushFeeder.queueForPush(article); + } + + private FeedManager() {} + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/feed/PullFeeder.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/feed/PullFeeder.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,250 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.feed; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.PrintWriter; +import java.net.Socket; +import java.net.SocketException; +import java.net.UnknownHostException; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import org.sonews.daemon.Config; +import org.sonews.util.Log; +import org.sonews.daemon.storage.Database; +import org.sonews.util.Stats; +import org.sonews.util.io.ArticleReader; +import org.sonews.util.io.ArticleWriter; + +/** + * The PullFeeder class regularily checks another Newsserver for new + * messages. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class PullFeeder extends AbstractFeeder +{ + + private Map highMarks = new HashMap(); + private BufferedReader in; + private PrintWriter out; + + @Override + public void addSubscription(final Subscription sub) + { + super.addSubscription(sub); + + // Set a initial highMark + this.highMarks.put(sub, 0); + } + + /** + * Changes to the given group and returns its high mark. + * @param groupName + * @return + */ + private int changeGroup(String groupName) + throws IOException + { + this.out.print("GROUP " + groupName + "\r\n"); + this.out.flush(); + + String line = this.in.readLine(); + if(line.startsWith("211 ")) + { + int highmark = Integer.parseInt(line.split(" ")[3]); + return highmark; + } + else + { + throw new IOException("GROUP " + groupName + " returned: " + line); + } + } + + private void connectTo(final String host, final int port) + throws IOException, UnknownHostException + { + Socket socket = new Socket(host, port); + this.out = new PrintWriter(socket.getOutputStream()); + this.in = new BufferedReader(new InputStreamReader(socket.getInputStream())); + + String line = in.readLine(); + if(!(line.charAt(0) == '2')) // Could be 200 or 2xx if posting is not allowed + { + throw new IOException(line); + } + } + + private void disconnect() + throws IOException + { + this.out.print("QUIT\r\n"); + this.out.flush(); + this.out.close(); + this.in.close(); + + this.out = null; + this.in = null; + } + + /** + * Uses the OVER or XOVER command to get a list of message overviews that + * may be unknown to this feeder and are about to be peered. + * @param start + * @param end + * @return A list of message ids with potentially interesting messages. + */ + private List over(int start, int end) + throws IOException + { + this.out.print("OVER " + start + "-" + end + "\r\n"); + this.out.flush(); + + String line = this.in.readLine(); + if(line.startsWith("500 ")) // OVER not supported + { + this.out.print("XOVER " + start + "-" + end + "\r\n"); + this.out.flush(); + + line = this.in.readLine(); + } + + if(line.startsWith("224 ")) + { + List messages = new ArrayList(); + line = this.in.readLine(); + while(!".".equals(line)) + { + String mid = line.split("\t")[4]; // 5th should be the Message-ID + messages.add(mid); + line = this.in.readLine(); + } + return messages; + } + else + { + throw new IOException("Server return for OVER/XOVER: " + line); + } + } + + @Override + public void run() + { + while(isRunning()) + { + int pullInterval = 1000 * + Config.getInstance().get(Config.FEED_PULLINTERVAL, 3600); + String host = "localhost"; + int port = 119; + + Log.msg("Start PullFeeder run...", true); + + try + { + for(Subscription sub : this.subscriptions) + { + host = sub.getHost(); + port = sub.getPort(); + + try + { + Log.msg("Feeding " + sub.getGroup() + " from " + sub.getHost(), true); + try + { + connectTo(host, port); + } + catch(SocketException ex) + { + Log.msg("Skipping " + sub.getHost() + ": " + ex, false); + continue; + } + + int oldMark = this.highMarks.get(sub); + int newMark = changeGroup(sub.getGroup()); + + if(oldMark != newMark) + { + List messageIDs = over(oldMark, newMark); + + for(String messageID : messageIDs) + { + if(Database.getInstance().isArticleExisting(messageID)) + { + continue; + } + + try + { + // Post the message via common socket connection + ArticleReader aread = + new ArticleReader(sub.getHost(), sub.getPort(), messageID); + byte[] abuf = aread.getArticleData(); + if (abuf == null) + { + Log.msg("Could not feed " + messageID + " from " + sub.getHost(), true); + } + else + { + Log.msg("Feeding " + messageID, true); + ArticleWriter awrite = new ArticleWriter( + "localhost", Config.getInstance().get(Config.PORT, 119)); + awrite.writeArticle(abuf); + awrite.close(); + } + Stats.getInstance().mailFeeded(sub.getGroup()); + } + catch(IOException ex) + { + // There may be a temporary network failure + ex.printStackTrace(); + Log.msg("Skipping mail " + messageID + " due to exception.", false); + } + } // for(;;) + this.highMarks.put(sub, newMark); + } + + disconnect(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + catch(IOException ex) + { + ex.printStackTrace(); + Log.msg("PullFeeder run stopped due to exception.", false); + } + } // for(Subscription sub : subscriptions) + + Log.msg("PullFeeder run ended. Waiting " + pullInterval / 1000 + "s", true); + Thread.sleep(pullInterval); + } + catch(InterruptedException ex) + { + Log.msg(ex.getMessage(), false); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/feed/PushFeeder.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/feed/PushFeeder.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,107 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.feed; + +import java.io.IOException; +import java.util.concurrent.ConcurrentLinkedQueue; +import org.sonews.daemon.storage.Article; +import org.sonews.daemon.storage.Headers; +import org.sonews.util.Log; +import org.sonews.util.io.ArticleWriter; + +/** + * Pushes new articles to remote newsservers. This feeder sleeps until a new + * message is posted to the sonews instance. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class PushFeeder extends AbstractFeeder +{ + + private ConcurrentLinkedQueue
articleQueue = + new ConcurrentLinkedQueue
(); + + @Override + public void run() + { + while(isRunning()) + { + try + { + synchronized(this) + { + this.wait(); + } + + Article article = this.articleQueue.poll(); + String[] groups = article.getHeader(Headers.NEWSGROUPS)[0].split(","); + Log.msg("PushFeed: " + article.getMessageID(), true); + for(Subscription sub : this.subscriptions) + { + // Circle check + if(article.getHeader(Headers.PATH)[0].contains(sub.getHost())) + { + Log.msg(article.getMessageID() + " skipped for host " + + sub.getHost(), true); + continue; + } + + try + { + for(String group : groups) + { + if(sub.getGroup().equals(group)) + { + // Delete headers that may cause problems + article.removeHeader(Headers.NNTP_POSTING_DATE); + article.removeHeader(Headers.NNTP_POSTING_HOST); + article.removeHeader(Headers.X_COMPLAINTS_TO); + article.removeHeader(Headers.X_TRACE); + article.removeHeader(Headers.XREF); + + // POST the message to remote server + ArticleWriter awriter = new ArticleWriter(sub.getHost(), sub.getPort()); + awriter.writeArticle(article); + break; + } + } + } + catch(IOException ex) + { + Log.msg(ex, false); + } + } + } + catch(InterruptedException ex) + { + Log.msg("PushFeeder interrupted.", true); + } + } + } + + public void queueForPush(Article article) + { + this.articleQueue.add(article); + synchronized(this) + { + this.notifyAll(); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/feed/Subscription.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/feed/Subscription.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,63 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.feed; + +/** + * For every group that is synchronized with or from a remote newsserver + * a Subscription instance exists. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class Subscription +{ + + private String host; + private int port; + private int feedtype; + private String group; + + public Subscription(String host, int port, int feedtype, String group) + { + this.host = host; + this.port = port; + this.feedtype = feedtype; + this.group = group; + } + + public int getFeedtype() + { + return feedtype; + } + + public String getGroup() + { + return group; + } + + public String getHost() + { + return host; + } + + public int getPort() + { + return port; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/feed/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/feed/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,2 @@ +Contains classes for the peering functionality, e.g. pulling and pushing +mails from and to remote newsservers. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/mlgw/Dispatcher.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/mlgw/Dispatcher.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,251 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.mlgw; + +import java.io.IOException; +import org.sonews.daemon.Config; +import org.sonews.daemon.storage.Article; +import org.sonews.util.io.ArticleInputStream; +import org.sonews.daemon.storage.Database; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.List; +import java.util.Properties; +import javax.mail.Address; +import javax.mail.Authenticator; +import javax.mail.Message; +import javax.mail.MessagingException; +import javax.mail.PasswordAuthentication; +import javax.mail.Session; +import javax.mail.Transport; +import javax.mail.internet.InternetAddress; +import javax.mail.internet.MimeMessage; +import org.sonews.daemon.storage.Headers; +import org.sonews.util.Log; +import org.sonews.util.Stats; + +/** + * Dispatches messages from mailing list or newsserver or vice versa. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class Dispatcher +{ + + static class PasswordAuthenticator extends Authenticator + { + + @Override + public PasswordAuthentication getPasswordAuthentication() + { + final String username = + Config.getInstance().get(Config.MLSEND_USER, "user"); + final String password = + Config.getInstance().get(Config.MLSEND_PASSWORD, "mysecret"); + + return new PasswordAuthentication(username, password); + } + + } + + /** + * Posts a message that was received from a mailing list to the + * appropriate newsgroup. + * @param msg + */ + public static boolean toGroup(final Message msg) + { + try + { + Address[] to = msg.getAllRecipients(); // includes TO/CC/BCC + if(to == null || to.length <= 0) + { + Log.msg("Skipping message because no receipient!", true); + return false; + } + else + { + boolean posted = false; + for(Address toa : to) // Address can have '<' '>' around + { + if(!(toa instanceof InternetAddress)) + { + continue; + } + String group = Database.getInstance() + .getGroupForList((InternetAddress)toa); + if(group != null) + { + Log.msg("Posting to group " + group, true); + + // Create new Article object + Article article = new Article(msg); + article.setGroup(group); + + // Write article to database + if(!Database.getInstance().isArticleExisting(article.getMessageID())) + { + Database.getInstance().addArticle(article); + Stats.getInstance().mailGatewayed( + article.getHeader(Headers.NEWSGROUPS)[0]); + } + else + { + Log.msg("Article " + article.getMessageID() + " already existing.", true); + // TODO: It may be possible that a ML mail is posted to several + // ML addresses... + } + posted = true; + } + else + { + Log.msg("No group for " + toa, true); + } + } // end for + return posted; + } + } + catch(Exception ex) + { + ex.printStackTrace(); + return false; + } + } + + /** + * Mails a message received through NNTP to the appropriate mailing list. + */ + public static void toList(Article article) + throws IOException, MessagingException, SQLException + { + // Get mailing lists for the group of this article + List listAddresses = new ArrayList(); + String[] groupnames = article.getHeader(Headers.NEWSGROUPS)[0].split(","); + + for(String groupname : groupnames) + { + String listAddress = Database.getInstance().getListForGroup(groupname); + if(listAddress != null) + { + listAddresses.add(listAddress); + } + } + + for(String listAddress : listAddresses) + { + // Compose message and send it via given SMTP-Host + String smtpHost = Config.getInstance().get(Config.MLSEND_HOST, "localhost"); + int smtpPort = Config.getInstance().get(Config.MLSEND_PORT, 25); + String smtpUser = Config.getInstance().get(Config.MLSEND_USER, "user"); + String smtpPw = Config.getInstance().get(Config.MLSEND_PASSWORD, "mysecret"); + + Properties props = System.getProperties(); + props.put("mail.smtp.localhost", + Config.getInstance().get(Config.HOSTNAME, "localhost")); + props.put("mail.smtp.from", // Used for MAIL FROM command + Config.getInstance().get( + Config.MLSEND_ADDRESS, article.getHeader(Headers.FROM)[0])); + props.put("mail.smtp.host", smtpHost); + props.put("mail.smtp.port", smtpPort); + props.put("mail.smtp.auth", "true"); + + Address[] address = new Address[1]; + address[0] = new InternetAddress(listAddress); + + ArticleInputStream in = new ArticleInputStream(article); + Session session = Session.getDefaultInstance(props, new PasswordAuthenticator()); + MimeMessage msg = new MimeMessage(session, in); + msg.setRecipient(Message.RecipientType.TO, address[0]); + msg.setReplyTo(address); + msg.removeHeader(Headers.NEWSGROUPS); + msg.removeHeader(Headers.PATH); + msg.removeHeader(Headers.LINES); + msg.removeHeader(Headers.BYTES); + + if(Config.getInstance().get(Config.MLSEND_RW_SENDER, false)) + { + rewriteSenderAddress(msg); // Set the SENDER address + } + + if(Config.getInstance().get(Config.MLSEND_RW_FROM, false)) + { + rewriteFromAddress(msg); // Set the FROM address + } + + msg.saveChanges(); + + // Send the mail + Transport transport = session.getTransport("smtp"); + transport.connect(smtpHost, smtpPort, smtpUser, smtpPw); + transport.sendMessage(msg, msg.getAllRecipients()); + transport.close(); + + Stats.getInstance().mailGatewayed(article.getHeader(Headers.NEWSGROUPS)[0]); + Log.msg("MLGateway: Mail " + article.getHeader("Subject")[0] + + " was delivered to " + listAddress + ".", true); + } + } + + /** + * Sets the SENDER header of the given MimeMessage. This might be necessary + * for moderated groups that does not allow the "normal" FROM sender. + * @param msg + * @throws javax.mail.MessagingException + */ + private static void rewriteSenderAddress(MimeMessage msg) + throws MessagingException + { + String mlAddress = Config.getInstance().get(Config.MLSEND_ADDRESS, null); + + if(mlAddress != null) + { + msg.setSender(new InternetAddress(mlAddress)); + } + else + { + throw new MessagingException("Cannot rewrite SENDER header!"); + } + } + + /** + * Sets the FROM header of the given MimeMessage. This might be necessary + * for moderated groups that does not allow the "normal" FROM sender. + * @param msg + * @throws javax.mail.MessagingException + */ + private static void rewriteFromAddress(MimeMessage msg) + throws MessagingException + { + Address[] froms = msg.getFrom(); + String mlAddress = Config.getInstance().get(Config.MLSEND_ADDRESS, null); + + if(froms.length > 0 && froms[0] instanceof InternetAddress + && mlAddress != null) + { + InternetAddress from = (InternetAddress)froms[0]; + from.setAddress(mlAddress); + msg.setFrom(from); + } + else + { + throw new MessagingException("Cannot rewrite FROM header!"); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/mlgw/MailPoller.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/mlgw/MailPoller.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,152 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.mlgw; + +import java.util.Properties; +import javax.mail.AuthenticationFailedException; +import javax.mail.Authenticator; +import javax.mail.Flags.Flag; +import javax.mail.Folder; +import javax.mail.Message; +import javax.mail.MessagingException; +import javax.mail.NoSuchProviderException; +import javax.mail.PasswordAuthentication; +import javax.mail.Session; +import javax.mail.Store; +import org.sonews.daemon.Config; +import org.sonews.daemon.AbstractDaemon; +import org.sonews.util.Log; +import org.sonews.util.Stats; + +/** + * Daemon polling for new mails in a POP3 account to be delivered to newsgroups. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class MailPoller extends AbstractDaemon +{ + + static class PasswordAuthenticator extends Authenticator + { + + @Override + public PasswordAuthentication getPasswordAuthentication() + { + final String username = + Config.getInstance().get(Config.MLPOLL_USER, "user"); + final String password = + Config.getInstance().get(Config.MLPOLL_PASSWORD, "mysecret"); + + return new PasswordAuthentication(username, password); + } + + } + + @Override + public void run() + { + Log.msg("Starting Mailinglist Poller...", false); + int errors = 0; + while(isRunning() && errors < 5) + { + try + { + // Wait some time between runs. At the beginning has advantages, + // because the wait is not skipped if an exception occurs. + Thread.sleep(60000 * (errors + 1)); // one minute * errors + + final String host = + Config.getInstance().get(Config.MLPOLL_HOST, "samplehost"); + final String username = + Config.getInstance().get(Config.MLPOLL_USER, "user"); + final String password = + Config.getInstance().get(Config.MLPOLL_PASSWORD, "mysecret"); + + Stats.getInstance().mlgwRunStart(); + + // Create empty properties + Properties props = System.getProperties(); + props.put("mail.pop3.host", host); + + // Get session + Session session = Session.getInstance(props); + + // Get the store + Store store = session.getStore("pop3"); + store.connect(host, 110, username, password); + + // Get folder + Folder folder = store.getFolder("INBOX"); + folder.open(Folder.READ_WRITE); + + // Get directory + Message[] messages = folder.getMessages(); + + // Dispatch messages and delete it afterwards on the inbox + for(Message message : messages) + { + String subject = message.getSubject(); + System.out.println("MLGateway: message with subject \"" + subject + "\" received."); + if(Dispatcher.toGroup(message) + || Config.getInstance().get(Config.MLPOLL_DELETEUNKNOWN, false)) + { + // Delete the message + message.setFlag(Flag.DELETED, true); + } + } + + // Close connection + folder.close(true); // true to expunge deleted messages + store.close(); + errors = 0; + + Stats.getInstance().mlgwRunEnd(); + } + catch(NoSuchProviderException ex) + { + Log.msg(ex.toString(), false); + shutdown(); + } + catch(AuthenticationFailedException ex) + { + // AuthentificationFailedException may be thrown if credentials are + // bad or if the Mailbox is in use (locked). + ex.printStackTrace(); + errors++; + } + catch(InterruptedException ex) + { + System.out.println("sonews: " + this + " returns."); + return; + } + catch(MessagingException ex) + { + ex.printStackTrace(); + errors++; + } + catch(Exception ex) + { + ex.printStackTrace(); + errors++; + } + } + Log.msg("MailPoller exited.", false); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/mlgw/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/mlgw/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains classes of the Mailinglist Gateway. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/AbstractConfig.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/AbstractConfig.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,43 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +/** + * Base class for Config and BootstrapConfig. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public abstract class AbstractConfig +{ + + public abstract String get(String key, String defVal); + + public int get(final String key, final int defVal) + { + return Integer.parseInt( + get(key, Integer.toString(defVal))); + } + + public boolean get(String key, boolean defVal) + { + String val = get(key, Boolean.toString(defVal)); + return Boolean.parseBoolean(val); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/DatabaseSetup.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/DatabaseSetup.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,130 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +import java.io.BufferedReader; +import java.io.InputStreamReader; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.Statement; +import java.util.HashMap; +import java.util.Map; +import org.sonews.daemon.BootstrapConfig; +import org.sonews.util.io.Resource; + +/** + * Database setup utility class. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class DatabaseSetup +{ + + private static final Map templateMap + = new HashMap(); + private static final Map urlMap + = new HashMap(); + private static final Map driverMap + = new HashMap(); + + static + { + templateMap.put("1", "helpers/database_mysql5_tmpl.sql"); + templateMap.put("2", "helpers/database_postgresql8_tmpl.sql"); + + urlMap.put("1", new StringTemplate("jdbc:mysql://%HOSTNAME/%DB")); + urlMap.put("2", new StringTemplate("jdbc:postgresql://%HOSTNAME/%DB")); + + driverMap.put("1", "com.mysql.jdbc.Driver"); + driverMap.put("2", "org.postgresql.Driver"); + } + + public static void main(String[] args) + throws Exception + { + System.out.println("sonews Database setup helper"); + System.out.println("This program will create a initial database table structure"); + System.out.println("for the sonews Newsserver."); + System.out.println("You need to create a database and a db user manually before!"); + + System.out.println("Select DBMS type:"); + System.out.println("[1] MySQL 5.x or higher"); + System.out.println("[2] PostgreSQL 8.x or higher"); + System.out.print("Your choice: "); + + BufferedReader in = new BufferedReader(new InputStreamReader(System.in)); + String dbmsType = in.readLine(); + String tmplName = templateMap.get(dbmsType); + if(tmplName == null) + { + System.err.println("Invalid choice. Try again you fool!"); + main(args); + return; + } + + // Load JDBC Driver class + Class.forName(driverMap.get(dbmsType)); + + String tmpl = Resource.getAsString(tmplName, true); + + System.out.print("Database server hostname (e.g. localhost): "); + String dbHostname = in.readLine(); + + System.out.print("Database name: "); + String dbName = in.readLine(); + + System.out.print("Give name of DB user that can create tables: "); + String dbUser = in.readLine(); + + System.out.print("Password: "); + String dbPassword = in.readLine(); + + String url = urlMap.get(dbmsType) + .set("HOSTNAME", dbHostname) + .set("DB", dbName).toString(); + + Connection conn = + DriverManager.getConnection(url, dbUser, dbPassword); + conn.setAutoCommit(false); + + String[] tmplChunks = tmpl.split(";"); + + for(String chunk : tmplChunks) + { + if(chunk.trim().equals("")) + continue; + + Statement stmt = conn.createStatement(); + stmt.execute(chunk); + } + + conn.commit(); + conn.setAutoCommit(true); + + BootstrapConfig config = BootstrapConfig.getInstance(); + config.set(BootstrapConfig.STORAGE_DATABASE, url); + config.set(BootstrapConfig.STORAGE_DBMSDRIVER, driverMap.get(dbmsType)); + config.set(BootstrapConfig.STORAGE_PASSWORD, dbPassword); + config.set(BootstrapConfig.STORAGE_USER, dbUser); + config.save(); + + System.out.println("Ok"); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/Log.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/Log.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,59 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +import org.sonews.daemon.*; +import java.util.Date; + +/** + * Provides logging and debugging methods. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class Log +{ + + public static boolean isDebug() + { + // We must use BootstrapConfig here otherwise we come + // into hell's kittchen when using the Logger within the + // Database class. + return BootstrapConfig.getInstance().get(BootstrapConfig.DEBUG, false); + } + + /** + * Writes the given message to the debug output. + * @param msg A String message or an object. + * @param If true this message is only shown if debug mode is enabled. + */ + public static void msg(final Object msg, boolean debug) + { + if(isDebug() || !debug) + { + synchronized(System.out) + { + System.out.print(new Date().toString()); + System.out.print(": "); + System.out.println(msg); + System.out.flush(); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/Pair.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/Pair.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,48 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +/** + * A pair of two objects. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class Pair +{ + + private T1 a; + private T2 b; + + public Pair(T1 a, T2 b) + { + this.a = a; + this.b = b; + } + + public T1 getA() + { + return a; + } + + public T2 getB() + { + return b; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/Purger.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/Purger.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,88 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +import org.sonews.daemon.Config; +import org.sonews.daemon.storage.Database; +import org.sonews.daemon.storage.Article; +import java.util.Date; + +/** + * The purger is started in configurable intervals to search + * for old messages that can be purged. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class Purger +{ + + private long lifetime; + + public Purger() + { + this.lifetime = Config.getInstance().get("sonews.article.lifetime", 30) + * 24L * 60L * 60L * 1000L; // in Milliseconds + } + + /** + * Loops through all messages and deletes them if their time + * has come. + */ + void purge() + throws Exception + { + System.out.println("Purging old messages..."); + + for (;;) + { + // TODO: Delete articles directly in database + Article art = null; //Database.getInstance().getOldestArticle(); + if (art == null) // No articles in the database + { + break; + } + +/* if (art.getDate().getTime() < (new Date().getTime() + this.lifetime)) + { + // Database.getInstance().delete(art); + System.out.println("Deleted: " + art); + } + else + { + break; + }*/ + } + } + + public static void main(String[] args) + { + try + { + Purger purger = new Purger(); + purger.purge(); + System.exit(0); + } + catch(Exception ex) + { + ex.printStackTrace(); + System.exit(1); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/Stats.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/Stats.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,194 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +import java.sql.SQLException; +import java.util.Calendar; +import org.sonews.daemon.storage.Database; +import org.sonews.daemon.storage.Group; + +/** + * Class that capsulates statistical data gathering. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Stats +{ + + public static final byte CONNECTIONS = 1; + public static final byte POSTED_NEWS = 2; + public static final byte GATEWAYED_NEWS = 3; + public static final byte FEEDED_NEWS = 4; + public static final byte MLGW_RUNSTART = 5; + public static final byte MLGW_RUNEND = 6; + + private static Stats instance = new Stats(); + + public static Stats getInstance() + { + return Stats.instance; + } + + private Stats() {} + + private volatile int connectedClients = 0; + + private void addEvent(byte type, String groupname) + { + Group group = Group.getByName(groupname); + if(group != null) + { + try + { + Database.getInstance().addEvent( + System.currentTimeMillis(), type, group.getID()); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + else + { + Log.msg("Group " + groupname + " does not exist.", true); + } + } + + public void clientConnect() + { + this.connectedClients++; + } + + public void clientDisconnect() + { + this.connectedClients--; + } + + public int connectedClients() + { + return this.connectedClients; + } + + public int getNumberOfGroups() + { + try + { + return Database.getInstance().countGroups(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + return -1; + } + } + + public int getNumberOfNews() + { + try + { + return Database.getInstance().countArticles(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + return -1; + } + } + + public int getYesterdaysEvents(final byte eventType, final int hour, + final Group group) + { + // Determine the timestamp values for yesterday and the given hour + Calendar cal = Calendar.getInstance(); + int year = cal.get(Calendar.YEAR); + int month = cal.get(Calendar.MONTH); + int dayom = cal.get(Calendar.DAY_OF_MONTH) - 1; // Yesterday + + cal.set(year, month, dayom, hour, 0, 0); + long startTimestamp = cal.getTimeInMillis(); + + cal.set(year, month, dayom, hour + 1, 0, 0); + long endTimestamp = cal.getTimeInMillis(); + + try + { + return Database.getInstance() + .getEventsCount(eventType, startTimestamp, endTimestamp, group); + } + catch(SQLException ex) + { + ex.printStackTrace(); + return -1; + } + } + + public void mailPosted(String groupname) + { + addEvent(POSTED_NEWS, groupname); + } + + public void mailGatewayed(String groupname) + { + addEvent(GATEWAYED_NEWS, groupname); + } + + public void mailFeeded(String groupname) + { + addEvent(FEEDED_NEWS, groupname); + } + + public void mlgwRunStart() + { + addEvent(MLGW_RUNSTART, "control"); + } + + public void mlgwRunEnd() + { + addEvent(MLGW_RUNEND, "control"); + } + + private double perHour(int key, long gid) + { + try + { + return Database.getInstance().getNumberOfEventsPerHour(key, gid); + } + catch(SQLException ex) + { + ex.printStackTrace(); + return -1; + } + } + + public double postedPerHour(long gid) + { + return perHour(POSTED_NEWS, gid); + } + + public double gatewayedPerHour(long gid) + { + return perHour(GATEWAYED_NEWS, gid); + } + + public double feededPerHour(long gid) + { + return perHour(FEEDED_NEWS, gid); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/StringTemplate.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/StringTemplate.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,97 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +import java.util.HashMap; +import java.util.Map; + +/** + * Class that allows simple String template handling. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class StringTemplate +{ + + private String str = null; + private String templateDelimiter = "%"; + private Map templateValues = new HashMap(); + + public StringTemplate(String str, final String templateDelimiter) + { + if(str == null || templateDelimiter == null) + { + throw new IllegalArgumentException("null arguments not allowed"); + } + + this.str = str; + this.templateDelimiter = templateDelimiter; + } + + public StringTemplate(String str) + { + this(str, "%"); + } + + public StringTemplate set(String template, String value) + { + if(template == null || value == null) + { + throw new IllegalArgumentException("null arguments not allowed"); + } + + this.templateValues.put(template, value); + return this; + } + + public StringTemplate set(String template, long value) + { + return set(template, Long.toString(value)); + } + + public StringTemplate set(String template, double value) + { + return set(template, Double.toString(value)); + } + + public StringTemplate set(String template, Object obj) + { + if(template == null || obj == null) + { + throw new IllegalArgumentException("null arguments not allowed"); + } + + return set(template, obj.toString()); + } + + @Override + public String toString() + { + String ret = str; + + for(String key : this.templateValues.keySet()) + { + String value = this.templateValues.get(key); + ret = ret.replace(templateDelimiter + key, value); + } + + return ret; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/TimeoutMap.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/TimeoutMap.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,145 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util; + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; + +/** + * Implementation of a Map that will loose its stored values after a + * configurable amount of time. + * This class may be used to cache config values for example. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class TimeoutMap extends ConcurrentHashMap +{ + + private static final long serialVersionUID = 453453467700345L; + + private int timeout = 60000; // 60 sec + private transient Map timeoutMap = new HashMap(); + + /** + * Constructor. + * @param timeout Timeout in milliseconds + */ + public TimeoutMap(final int timeout) + { + this.timeout = timeout; + } + + /** + * Uses default timeout (60 sec). + */ + public TimeoutMap() + { + } + + /** + * + * @param key + * @return true if key is still valid. + */ + protected boolean checkTimeOut(Object key) + { + synchronized(this.timeoutMap) + { + if(this.timeoutMap.containsKey(key)) + { + long keytime = this.timeoutMap.get(key); + if((System.currentTimeMillis() - keytime) < this.timeout) + { + return true; + } + else + { + remove(key); + return false; + } + } + else + { + return false; + } + } + } + + @Override + public boolean containsKey(Object key) + { + return checkTimeOut(key); + } + + @Override + public synchronized V get(Object key) + { + if(checkTimeOut(key)) + { + return super.get(key); + } + else + { + return null; + } + } + + @Override + public V put(K key, V value) + { + synchronized(this.timeoutMap) + { + removeStaleKeys(); + this.timeoutMap.put(key, System.currentTimeMillis()); + return super.put(key, value); + } + } + + /** + * @param arg0 + * @return + */ + @Override + public V remove(Object arg0) + { + synchronized(this.timeoutMap) + { + this.timeoutMap.remove(arg0); + V val = super.remove(arg0); + return val; + } + } + + protected void removeStaleKeys() + { + synchronized(this.timeoutMap) + { + Set keySet = new HashSet(this.timeoutMap.keySet()); + for(Object key : keySet) + { + // The key/value is removed by the checkTimeOut() method if true + checkTimeOut(key); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/io/ArticleInputStream.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/io/ArticleInputStream.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,61 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util.io; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import org.sonews.daemon.storage.*; +import java.io.InputStream; +import java.io.UnsupportedEncodingException; + +/** + * Capsulates an Article to provide a raw InputStream. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ArticleInputStream extends InputStream +{ + + private byte[] buffer; + private int offset = 0; + + public ArticleInputStream(final Article art) + throws IOException, UnsupportedEncodingException + { + final ByteArrayOutputStream out = new ByteArrayOutputStream(); + out.write(art.getHeaderSource().getBytes("UTF-8")); + out.write("\r\n\r\n".getBytes()); + out.write(art.getBody().getBytes(art.getBodyCharset())); + out.flush(); + this.buffer = out.toByteArray(); + } + + public int read() + { + if(offset >= buffer.length) + { + return -1; + } + else + { + return buffer[offset++]; + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/io/ArticleReader.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/io/ArticleReader.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,127 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util.io; + +import java.io.BufferedInputStream; +import java.io.BufferedOutputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.UnsupportedEncodingException; +import java.net.Socket; +import java.net.UnknownHostException; +import org.sonews.util.Log; + +/** + * Reads an news article from a NNTP server. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ArticleReader +{ + + private BufferedOutputStream out; + private BufferedInputStream in; + private String messageID; + + public ArticleReader(String host, int port, String messageID) + throws IOException, UnknownHostException + { + this.messageID = messageID; + + // Connect to NNTP server + Socket socket = new Socket(host, port); + this.out = new BufferedOutputStream(socket.getOutputStream()); + this.in = new BufferedInputStream(socket.getInputStream()); + String line = readln(this.in); + if(!line.startsWith("200 ")) + { + throw new IOException("Invalid hello from server: " + line); + } + } + + private boolean eofArticle(byte[] buf) + { + if(buf.length < 4) + { + return false; + } + + int l = buf.length - 1; + return buf[l-3] == 10 // '*\n' + && buf[l-2] == '.' // '.' + && buf[l-1] == 13 && buf[l] == 10; // '\r\n' + } + + public byte[] getArticleData() + throws IOException, UnsupportedEncodingException + { + try + { + this.out.write(("ARTICLE " + this.messageID + "\r\n").getBytes("UTF-8")); + this.out.flush(); + + String line = readln(this.in); + if(line.startsWith("220 ")) + { + ByteArrayOutputStream buf = new ByteArrayOutputStream(); + + while(!eofArticle(buf.toByteArray())) + { + for(int b = in.read(); b != 10; b = in.read()) + { + buf.write(b); + } + + buf.write(10); + } + + return buf.toByteArray(); + } + else + { + Log.msg("ArticleReader: " + line, false); + return null; + } + } + catch(IOException ex) + { + throw ex; + } + finally + { + this.out.write("QUIT\r\n".getBytes("UTF-8")); + this.out.flush(); + this.out.close(); + } + } + + private String readln(InputStream in) + throws IOException + { + ByteArrayOutputStream buf = new ByteArrayOutputStream(); + for(int b = in.read(); b != 10 /* \n */; b = in.read()) + { + buf.write(b); + } + + return new String(buf.toByteArray()); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/io/ArticleWriter.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/io/ArticleWriter.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,133 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util.io; + +import java.io.BufferedOutputStream; +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.UnsupportedEncodingException; +import java.net.Socket; +import java.net.UnknownHostException; +import org.sonews.daemon.storage.Article; + +/** + * Posts an Article to a NNTP server using the POST command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ArticleWriter +{ + + private BufferedOutputStream out; + private BufferedReader inr; + + public ArticleWriter(String host, int port) + throws IOException, UnknownHostException + { + // Connect to NNTP server + Socket socket = new Socket(host, port); + this.out = new BufferedOutputStream(socket.getOutputStream()); + this.inr = new BufferedReader(new InputStreamReader(socket.getInputStream())); + String line = inr.readLine(); + if(line == null || !line.startsWith("200 ")) + { + throw new IOException("Invalid hello from server: " + line); + } + } + + public void close() + throws IOException, UnsupportedEncodingException + { + this.out.write("QUIT\r\n".getBytes("UTF-8")); + this.out.flush(); + } + + protected void finishPOST() + throws IOException + { + this.out.write("\r\n.\r\n".getBytes()); + this.out.flush(); + String line = inr.readLine(); + if(line == null || !line.startsWith("240 ")) + { + throw new IOException(line); + } + } + + protected void preparePOST() + throws IOException + { + this.out.write("POST\r\n".getBytes("UTF-8")); + this.out.flush(); + + String line = this.inr.readLine(); + if(line == null || !line.startsWith("340 ")) + { + throw new IOException(line); + } + } + + public void writeArticle(Article article) + throws IOException, UnsupportedEncodingException + { + byte[] buf = new byte[512]; + ArticleInputStream in = new ArticleInputStream(article); + + preparePOST(); + + int len = in.read(buf); + while(len != -1) + { + writeLine(buf, len); + len = in.read(buf); + } + + finishPOST(); + } + + /** + * Writes the raw content of an article to the remote server. This method + * does no charset conversion/handling of any kind so its the preferred + * method for sending an article to remote peers. + * @param rawArticle + * @throws IOException + */ + public void writeArticle(byte[] rawArticle) + throws IOException + { + preparePOST(); + writeLine(rawArticle, rawArticle.length); + finishPOST(); + } + + /** + * Writes the given buffer to the connect remote server. + * @param buffer + * @param len + * @throws IOException + */ + protected void writeLine(byte[] buffer, int len) + throws IOException + { + this.out.write(buffer, 0, len); + this.out.flush(); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/io/Resource.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/io/Resource.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,132 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util.io; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.net.URL; +import java.nio.charset.Charset; + +/** + * Provides method for loading of resources. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Resource +{ + + /** + * Loads a resource and returns it as URL reference. + * The Resource's classloader is used to load the resource, not + * the System's ClassLoader so it may be safe to use this method + * in a sandboxed environment. + * @return + */ + public static URL getAsURL(final String name) + { + if(name == null) + { + return null; + } + + return Resource.class.getClassLoader().getResource(name); + } + + /** + * Loads a resource and returns an InputStream to it. + * @param name + * @return + */ + public static InputStream getAsStream(String name) + { + try + { + URL url = getAsURL(name); + if(url == null) + { + return null; + } + else + { + return url.openStream(); + } + } + catch(IOException e) + { + e.printStackTrace(); + return null; + } + } + + /** + * Loads a plain text resource. + * @param withNewline If false all newlines are removed from the + * return String + */ + public static String getAsString(String name, boolean withNewline) + { + if(name == null) + return null; + + BufferedReader in = null; + try + { + InputStream ins = getAsStream(name); + if(ins == null) + return null; + + in = new BufferedReader( + new InputStreamReader(ins, Charset.forName("UTF-8"))); + StringBuffer buf = new StringBuffer(); + + for(;;) + { + String line = in.readLine(); + if(line == null) + break; + + buf.append(line); + if(withNewline) + buf.append('\n'); + } + + return buf.toString(); + } + catch(Exception e) + { + e.printStackTrace(); + return null; + } + finally + { + try + { + if(in != null) + in.close(); + } + catch(IOException ex) + { + ex.printStackTrace(); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/io/VarCharsetReader.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/io/VarCharsetReader.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,90 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.util.io; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.ByteBuffer; +import java.nio.charset.Charset; + +/** + * InputStream that can change its decoding charset while reading from the + * underlying byte based stream. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class VarCharsetReader +{ + + private final ByteBuffer buf = ByteBuffer.allocate(4096); + private InputStream in; + + public VarCharsetReader(final InputStream in) + { + if(in == null) + { + throw new IllegalArgumentException("null InputStream"); + } + this.in = in; + } + + /** + * Reads up to the next newline character and returns the line as String. + * The String is decoded using the given charset. + */ + public String readLine(Charset charset) + throws IOException + { + byte[] byteBuf = new byte[1]; + String bufStr; + + for(;;) + { + int read = this.in.read(byteBuf); + if(read == 0) + { + continue; + } + else if(read == -1) + { + this.in = null; + bufStr = new String(this.buf.array(), 0, this.buf.position(), charset); + break; + } + else if(byteBuf[0] == 10) // Is this safe? \n + { + bufStr = new String(this.buf.array(), 0, this.buf.position(), charset); + break; + } + else if(byteBuf[0] == 13) // \r + { // Skip + continue; + } + else + { + this.buf.put(byteBuf[0]); + } + } + + this.buf.clear(); + + return bufStr; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/io/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/io/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains I/O utilitiy classes. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/util/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/util/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains various utility classes. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/AbstractSonewsServlet.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/AbstractSonewsServlet.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,113 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.PrintWriter; +import java.net.Socket; +import javax.servlet.http.HttpServlet; +import org.sonews.util.StringTemplate; +import org.sonews.util.io.Resource; + +/** + * Base class for all sonews servlets. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class AbstractSonewsServlet extends HttpServlet +{ + + public static final String TemplateRoot = "org/sonews/web/tmpl/"; + + protected String hello = null; + + private BufferedReader in = null; + private PrintWriter out = null; + private Socket socket = null; + + protected void connectToNewsserver() + throws IOException + { + // Get sonews port from properties + String port = System.getProperty("sonews.port", "9119"); + String host = System.getProperty("sonews.host", "localhost"); + + try + { + this.socket = new Socket(host, Integer.parseInt(port)); + + this.in = new BufferedReader( + new InputStreamReader(socket.getInputStream())); + this.out = new PrintWriter(socket.getOutputStream()); + + hello = in.readLine(); // Read hello message + } + catch(IOException ex) + { + System.out.println("sonews.host=" + host); + System.out.println("sonews.port=" + port); + System.out.flush(); + throw ex; + } + } + + protected void disconnectFromNewsserver() + { + try + { + printlnToNewsserver("QUIT"); + out.close(); + readlnFromNewsserver(); // Wait for bye message + in.close(); + socket.close(); + } + catch(IOException ex) + { + ex.printStackTrace(); + } + } + + protected StringTemplate getTemplate(String res) + { + StringTemplate tmpl = new StringTemplate( + Resource.getAsString(TemplateRoot + "AbstractSonewsServlet.tmpl", true)); + String content = Resource.getAsString(TemplateRoot + res, true); + String stylesheet = System.getProperty("sonews.web.stylesheet", "style.css"); + + tmpl.set("CONTENT", content); + tmpl.set("STYLESHEET", stylesheet); + + return new StringTemplate(tmpl.toString()); + } + + protected void printlnToNewsserver(final String line) + { + this.out.println(line); + this.out.flush(); + } + + protected String readlnFromNewsserver() + throws IOException + { + return this.in.readLine(); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/MemoryBitmapChart.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/MemoryBitmapChart.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,61 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import info.monitorenter.gui.chart.Chart2D; +import info.monitorenter.gui.chart.IAxis.AxisTitle; +import java.awt.Color; +import java.awt.image.BufferedImage; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import javax.imageio.ImageIO; + +/** + * A chart rendered to a memory bitmap. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class MemoryBitmapChart extends Chart2D +{ + + public MemoryBitmapChart() + { + setGridColor(Color.LIGHT_GRAY); + getAxisX().setPaintGrid(true); + getAxisY().setPaintGrid(true); + getAxisX().setAxisTitle(new AxisTitle("time of day")); + getAxisY().setAxisTitle(new AxisTitle("processed news")); + } + + public String getContentType() + { + return "image/png"; + } + + public byte[] getRawData(final int width, final int height) + throws IOException + { + setSize(width, height); + BufferedImage img = snapShot(width, height); + ByteArrayOutputStream out = new ByteArrayOutputStream(); + ImageIO.write(img, "png", out); + return out.toByteArray(); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/SonewsChartServlet.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/SonewsChartServlet.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,114 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import info.monitorenter.gui.chart.ITrace2D; +import info.monitorenter.gui.chart.traces.Trace2DSimple; +import java.io.IOException; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; + +/** + * Servlet that creates chart images and returns them as raw PNG images. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class SonewsChartServlet extends AbstractSonewsServlet +{ + + private ITrace2D createProcessMails24(String title, String cmd) + throws IOException + { + int[] data = read24Values(cmd); + ITrace2D trace = new Trace2DSimple(title); + trace.addPoint(0.0, 0.0); // Start + + for(int n = 0; n < 24; n++) + { + trace.addPoint(n, data[n]); + } + + return trace; + } + + @Override + public void doGet(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + synchronized(this) + { + MemoryBitmapChart chart = new MemoryBitmapChart(); + + String name = req.getParameter("name"); + String group = req.getParameter("group"); + ITrace2D trace; + String cmd = "XDAEMON LOG"; + + if(name.equals("feedednewsyesterday")) + { + cmd = cmd + " TRANSMITTED_NEWS"; + cmd = group != null ? cmd + " " + group : cmd; + trace = createProcessMails24( + "To peers transmitted mails yesterday", cmd); + } + else if(name.equals("gatewayednewsyesterday")) + { + cmd = cmd + " GATEWAYED_NEWS"; + cmd = group != null ? cmd + " " + group : cmd; + trace = createProcessMails24( + "Gatewayed mails yesterday", cmd); + } + else + { + cmd = cmd + " POSTED_NEWS"; + cmd = group != null ? cmd + " " + group : cmd; + trace = createProcessMails24( + "Posted mails yesterday", cmd); + } + chart.addTrace(trace); + + resp.getOutputStream().write(chart.getRawData(500, 400)); + resp.setContentType(chart.getContentType()); + resp.setStatus(HttpServletResponse.SC_OK); + } + } + + private int[] read24Values(String command) + throws IOException + { + int[] values = new int[24]; + connectToNewsserver(); + printlnToNewsserver(command); + String line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + throw new IOException(command + " not supported!"); + + for(int n = 0; n < 24; n++) + { + line = readlnFromNewsserver(); + values[n] = Integer.parseInt(line.split(" ")[1]); + } + + line = readlnFromNewsserver(); // "." + + disconnectFromNewsserver(); + return values; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/SonewsConfigServlet.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/SonewsConfigServlet.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,239 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.Set; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import org.sonews.util.StringTemplate; +import org.sonews.util.io.Resource; + +/** + * Servlet providing a configuration web interface. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class SonewsConfigServlet extends AbstractSonewsServlet +{ + + private static final long serialVersionUID = 2432543253L; + + @Override + public void doGet(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + synchronized(this) + { + connectToNewsserver(); + String which = req.getParameter("which"); + + if(which != null && which.equals("config")) + { + whichConfig(req, resp); + } + else if(which != null && which.equals("groupadd")) + { + whichGroupAdd(req, resp); + } + else if(which != null && which.equals("groupdelete")) + { + whichGroupDelete(req, resp); + } + else + { + whichNone(req, resp); + } + + disconnectFromNewsserver(); + } + } + + private void whichConfig(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + StringBuilder keys = new StringBuilder(); + + Set pnames = req.getParameterMap().keySet(); + for(Object obj : pnames) + { + String pname = (String)obj; + if(pname.startsWith("configkey:")) + { + String value = req.getParameter(pname); + String key = pname.split(":")[1]; + if(!value.equals("")) + { + printlnToNewsserver("XDAEMON SET " + key + " " + value); + readlnFromNewsserver(); + + keys.append(key); + keys.append("
"); + } + } + } + + StringTemplate tmpl = getTemplate("ConfigUpdated.tmpl"); + + tmpl.set("UPDATED_KEYS", keys.toString()); + + resp.setStatus(HttpServletResponse.SC_OK); + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + } + + private void whichGroupAdd(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + String[] groupnames = req.getParameter("groups").split("\n"); + + for(String groupname : groupnames) + { + groupname = groupname.trim(); + if(groupname.equals("")) + { + continue; + } + + printlnToNewsserver("XDAEMON GROUPADD " + groupname + " 0"); + String line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + System.out.println("Warning " + groupname + " probably not created!"); + } + } + + StringTemplate tmpl = getTemplate("GroupAdded.tmpl"); + + tmpl.set("GROUP", req.getParameter("groups")); + + resp.setStatus(HttpServletResponse.SC_OK); + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + } + + private void whichGroupDelete(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + String groupname = req.getParameter("group"); + printlnToNewsserver("XDAEMON GROUPDEL " + groupname); + String line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + throw new IOException(line); + + StringTemplate tmpl = getTemplate("GroupDeleted.tmpl"); + + tmpl.set("GROUP", groupname); + + resp.setStatus(HttpServletResponse.SC_OK); + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + } + + private void whichNone(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + StringTemplate tmpl = getTemplate("SonewsConfigServlet.tmpl"); + + // Retrieve config keys from server + List configKeys = new ArrayList(); + printlnToNewsserver("XDAEMON LIST CONFIGKEYS"); + String line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + throw new IOException("XDAEMON command not supported!"); + for(;;) + { + line = readlnFromNewsserver(); + if(line.equals(".")) + break; + else + configKeys.add(line); + } + + // Construct config table + StringBuilder strb = new StringBuilder(); + for(String key : configKeys) + { + strb.append(""); + strb.append(key); + strb.append(""); + + // Retrieve config value from server + String value = ""; + printlnToNewsserver("XDAEMON GET " + key); + line = readlnFromNewsserver(); + if(line.startsWith("200 ")) + { + value = readlnFromNewsserver(); + readlnFromNewsserver(); // Read the "." + } + + strb.append(""); + } + tmpl.set("CONFIG", strb.toString()); + + // Retrieve served newsgroup names from server + List groups = new ArrayList(); + printlnToNewsserver("LIST"); + line = readlnFromNewsserver(); + if(line.startsWith("215 ")) + { + for(;;) + { + line = readlnFromNewsserver(); + if(line.equals(".")) + { + break; + } + else + { + groups.add(line.split(" ")[0]); + } + } + } + else + throw new IOException("Error issuing LIST command!"); + + // Construct groups list + StringTemplate tmplGroupList = new StringTemplate( + Resource.getAsString("org/sonews/web/tmpl/GroupList.tmpl", true)); + strb = new StringBuilder(); + for(String group : groups) + { + tmplGroupList.set("GROUPNAME", group); + strb.append(tmplGroupList.toString()); + } + tmpl.set("GROUP", strb.toString()); + + // Set server name + tmpl.set("SERVERNAME", hello.split(" ")[2]); + tmpl.set("TITLE", "Configuration"); + + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + resp.setStatus(HttpServletResponse.SC_OK); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/SonewsGroupServlet.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/SonewsGroupServlet.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,66 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import java.io.IOException; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import org.sonews.util.StringTemplate; + +/** + * Views the group settings and allows editing. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class SonewsGroupServlet extends AbstractSonewsServlet +{ + + @Override + public void doGet(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + synchronized(this) + { + connectToNewsserver(); + String name = req.getParameter("name"); + String action = req.getParameter("action"); + + if("set_flags".equals(action)) + { + + } + else if("set_mladdress".equals(action)) + { + + } + + StringTemplate tmpl = getTemplate("SonewsGroupServlet.tmpl"); + tmpl.set("SERVERNAME", hello.split(" ")[2]); + tmpl.set("TITLE", "Group " + name); + tmpl.set("GROUPNAME", name); + + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + resp.setStatus(HttpServletResponse.SC_OK); + + disconnectFromNewsserver(); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/SonewsPeerServlet.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/SonewsPeerServlet.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,95 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import java.io.IOException; +import java.util.HashSet; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import org.sonews.util.StringTemplate; + +/** + * Servlet that shows the Peers and the Peering Rules. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class SonewsPeerServlet extends AbstractSonewsServlet +{ + + private static final long serialVersionUID = 245345346356L; + + @Override + public void doGet(HttpServletRequest req, HttpServletResponse resp) + throws IOException + { + synchronized(this) + { + connectToNewsserver(); + StringTemplate tmpl = getTemplate("SonewsPeerServlet.tmpl"); + + // Read peering rules from newsserver + printlnToNewsserver("XDAEMON LIST PEERINGRULES"); + String line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("Unexpected reply: " + line); + } + + // Create FEED_RULES String + HashSet peers = new HashSet(); + StringBuilder feedRulesStr = new StringBuilder(); + for(;;) + { + line = readlnFromNewsserver(); + if(line.equals(".")) + { + break; + } + else + { + feedRulesStr.append(line); + feedRulesStr.append("
"); + + String[] lineChunks = line.split(" "); + peers.add(lineChunks[1]); + } + } + + // Create PEERS string + StringBuilder peersStr = new StringBuilder(); + for(String peer : peers) + { + peersStr.append(peer); + peersStr.append("
"); + } + + // Set server name + tmpl.set("PEERS", peersStr.toString()); + tmpl.set("PEERING_RULES", feedRulesStr.toString()); + tmpl.set("SERVERNAME", hello.split(" ")[2]); + tmpl.set("TITLE", "Peers"); + + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + resp.setStatus(HttpServletResponse.SC_OK); + disconnectFromNewsserver(); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/SonewsServlet.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/SonewsServlet.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,127 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.web; + +import java.io.IOException; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; +import org.sonews.daemon.Main; +import org.sonews.util.StringTemplate; + +/** + * Main sonews webpage servlet. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class SonewsServlet extends AbstractSonewsServlet +{ + + private static final long serialVersionUID = 2392837459834L; + + @Override + public void doGet(HttpServletRequest res, HttpServletResponse resp) + throws IOException + { + synchronized(this) + { + connectToNewsserver(); + + String line; + int connectedClients = 0; + int hostedGroups = 0; + int hostedNews = 0; + + printlnToNewsserver("XDAEMON LOG CONNECTED_CLIENTS"); + + line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("XDAEMON command not allowed by server"); + } + line = readlnFromNewsserver(); + connectedClients = Integer.parseInt(line); + line = readlnFromNewsserver(); // Read the "." + + printlnToNewsserver("XDAEMON LOG HOSTED_NEWS"); + line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("XDAEMON command not allowed by server"); + } + line = readlnFromNewsserver(); + hostedNews = Integer.parseInt(line); + line = readlnFromNewsserver(); // read the "." + + printlnToNewsserver("XDAEMON LOG HOSTED_GROUPS"); + line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("XDAEMON command not allowed by server"); + } + line = readlnFromNewsserver(); + hostedGroups = Integer.parseInt(line); + line = readlnFromNewsserver(); // read the "." + + printlnToNewsserver("XDAEMON LOG POSTED_NEWS_PER_HOUR"); + line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("XDAEMON command not allowed by server"); + } + String postedNewsPerHour = readlnFromNewsserver(); + readlnFromNewsserver(); + + printlnToNewsserver("XDAEMON LOG GATEWAYED_NEWS_PER_HOUR"); + line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("XDAEMON command not allowed by server"); + } + String gatewayedNewsPerHour = readlnFromNewsserver(); + line = readlnFromNewsserver(); + + printlnToNewsserver("XDAEMON LOG FEEDED_NEWS_PER_HOUR"); + line = readlnFromNewsserver(); + if(!line.startsWith("200 ")) + { + throw new IOException("XDAEMON command not allowed by server"); + } + String feededNewsPerHour = readlnFromNewsserver(); + line = readlnFromNewsserver(); + + StringTemplate tmpl = getTemplate("SonewsServlet.tmpl"); + tmpl.set("SERVERNAME", hello.split(" ")[2]); + tmpl.set("STARTDATE", Main.STARTDATE); + tmpl.set("ACTIVE_CONNECTIONS", connectedClients); + tmpl.set("STORED_NEWS", hostedNews); + tmpl.set("SERVED_NEWSGROUPS", hostedGroups); + tmpl.set("POSTED_NEWS", postedNewsPerHour); + tmpl.set("GATEWAYED_NEWS", gatewayedNewsPerHour); + tmpl.set("FEEDED_NEWS", feededNewsPerHour); + tmpl.set("TITLE", "Overview"); + + resp.getWriter().println(tmpl.toString()); + resp.getWriter().flush(); + resp.setStatus(HttpServletResponse.SC_OK); + + disconnectFromNewsserver(); + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,3 @@ +Contains classes of the sonews web interface. These classes are not needed by +the running sonews daemon but by the Servlet container +Kitten. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/AbstractSonewsServlet.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/AbstractSonewsServlet.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,14 @@ + + + %SERVERNAME - %TITLE + + + + +
sonews - %TITLE
+ +
+%CONTENT +
+ + \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/ConfigUpdated.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/ConfigUpdated.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,7 @@ +

+ The following config keys were updated:
+ %UPDATED_KEYS +

+

+ Back to Config +

\ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/GroupAdded.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/GroupAdded.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,6 @@ +

+ The Newsgroup %GROUP has been created! +

+

+ Back to Config +

\ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/GroupDeleted.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/GroupDeleted.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,6 @@ +

+ The Newsgroup %GROUP and all associated articles have been deleted! +

+

+ Back to Config +

\ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/GroupList.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/GroupList.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,8 @@ + + + %GROUPNAME + + + delete + + \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/SonewsConfigServlet.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/SonewsConfigServlet.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,30 @@ +

+Back to Main Page +

+ +

Configuration values

+
+ + +%CONFIG +
+ +
+ +

Groups served by this sonews instance

+ +%GROUP +
+ +

+

Add new group to be served

+
+ + + + + +
Names (separated by newlines):
+ +
+

\ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/SonewsGroupServlet.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/SonewsGroupServlet.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,36 @@ +

+Back to Main Page +

+ +

Group %GROUPNAME

+ +

Configuration

+

General

+
+ + + Is mirrored Mailinglist?: + +
+ +
+ +

Mailinglist

+
+ + + Mailinglist address: + +
+ +
+ +

Statistics

+

Posted mails yesterday

+ + +

Gatewayed mails yesterday

+ + +

Feeded news

+ \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/SonewsPeerServlet.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/SonewsPeerServlet.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,52 @@ +

+Back to Main Page +

+ +On this page you can configure to which peer hosts new messages are +posted or from which peer hosts we should receive messages. + +

Peers

+%PEERS + +

Add new peer

+
+ + + + + + + + + +
Hostname:
Port:
+ +
+ +

Rules

+

Current

+%PEERING_RULES + +

Add peering rule

+
+ + + + + + + + + + + + + +
Peer:%OPTION_PEERS
Group:%OPTION_GROUP
Peering type: + +
+ +
\ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 org/sonews/web/tmpl/SonewsServlet.tmpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/web/tmpl/SonewsServlet.tmpl Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,54 @@ +This server is running since %STARTDATE. + +

Configuration

+Here you can edit most of the configuration values for %SERVERNAME. +Please note that some of these values require a server restart, +some do not and are valid immediately. + + +

Statistics & Logs

+Here is a short overview of useful statistics of %SERVERNAME. Click on a +stat key to get more details. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Stat keyValue
Active connections:%ACTIVE_CONNECTIONS
Served newsgroups:%SERVED_NEWSGROUPS
Stored news messages:%STORED_NEWS
Posted news:%POSTED_NEWS per hour
Gatewayed news:%GATEWAYED_NEWS per hour
Feeded news:%FEEDED_NEWS per hour
+ +

Documentation

+You'll find the most recent +documentation of %SERVERNAME here. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 test/AbstractTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/AbstractTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,93 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test; + +import java.io.BufferedReader; +import java.io.File; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.PrintWriter; +import java.net.Socket; +import java.net.UnknownHostException; + +/** + * Base class for every test performed by the TestBench. + * Connects to a NNTP Server and provides basic methods for sending and + * receiving data. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public abstract class AbstractTest +{ + + protected static PrintWriter log; + + static + { + try + { + log = new PrintWriter(new File("test.log")); + } + catch(Exception ex) + { + ex.printStackTrace(); + } + } + + protected BufferedReader in; + protected PrintWriter out; + protected Socket socket; + + /** + * Connects to NNTP Server using for + * @param host + * @param port + * @throws java.io.IOException + * @throws java.net.UnknownHostException + */ + public void connect(String host, int port) + throws IOException, UnknownHostException + { + socket = new Socket(host, port); + socket.setSoTimeout(10000); + this.in = new BufferedReader(new InputStreamReader(socket.getInputStream())); + this.out = new PrintWriter(socket.getOutputStream()); + } + + protected void println(String line) + { + this.out.println(line); + this.out.flush(); + + log.println(">> " + line); + log.flush(); + } + + protected String readln() + throws IOException + { + String line = this.in.readLine(); + log.println("<< " + line); + log.flush(); + return line; + } + + public abstract int runTest() throws Exception; + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/CollectionsSpeedTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/CollectionsSpeedTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,72 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test; + +import java.util.ArrayList; +import java.util.LinkedList; +import java.util.List; +import java.util.ListIterator; + +/** + * Tests the speed of LinkedList and ArrayList. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class CollectionsSpeedTest +{ + + public static void main(String[] args) + { + List arrayList = new ArrayList(); + List linkedList = new LinkedList(); + + int numElements = 100000; + + System.out.println("ArrayList.add(): " + add(arrayList, numElements) + "ms"); + System.out.println("LinkenList.add(): " + add(linkedList, numElements) + "ms"); + System.out.println("ArrayList.iterate: " + iterate(arrayList) + "ms"); + System.out.println("LinkedList.iterate: " + iterate(linkedList) + "ms"); + } + + private static long add(List list, int numElements) + { + long start = System.currentTimeMillis(); + + for(int n = 0; n < numElements; n++) + { + list.add(new Object()); + } + + return System.currentTimeMillis() - start; + } + + private static long iterate(List list) + { + long start = System.currentTimeMillis(); + + ListIterator iter = list.listIterator(); + while(iter.hasNext()) + { + iter.next(); + } + + return System.currentTimeMillis() - start; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/PerfTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/PerfTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,44 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test; + +/** + * Opens a connection, waits for Hello and exits while leaving the connection + * open until SoTimeout. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class PerfTest extends AbstractTest +{ + + @Override + public int runTest() throws Exception + { + String line = readln(); + if(!line.startsWith("200 ")) + { + return 1; + } + + socket.setSoTimeout(0); + + return 0; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/PerfTestBench.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/PerfTestBench.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,63 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test; + +/** + * TestBench that performs a - yes - performance test. The test runs until + * the opened sockets reach the systems maximum. The test has now valid end + * as it will throw IOErrors at the end. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class PerfTestBench +{ + + public static void main(String[] args) + throws Exception + { + System.out.println("Performance TestBench for NNTP (RFC3799) based servers "); + if(args.length < 1) + { + System.out.println("Usage: TestBench [:port]"); + return; + } + + String[] hostport = args[0].split(":"); + String host = hostport[0]; + int port = 119; + if(hostport.length == 2) + { + port = Integer.parseInt(hostport[1]); + } + + for(int n = 0; true; n++) + { + PerfTest pf = new PerfTest(); + pf.connect(host, port); + pf.runTest(); + + if(n % 100 == 0) + { + System.out.println("Test #" + n); + System.out.flush(); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/StringTemplateTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/StringTemplateTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,43 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test; + +import org.sonews.util.StringTemplate; + +/** + * Tests the StringTemplate class. + * @author Christian Lins + * @since sonews/0.5.0 + * @see org.sonews.util.StringTemplate + */ +public class StringTemplateTest +{ + + public static void main(String[] args) + { + StringTemplate templ + = new StringTemplate("SELECT %row FROM %table WHERE %row = ich"); + + templ.set("row", "name"); + templ.set("table", "UserTable"); + + System.out.println(templ.toString()); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/TestBench.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/TestBench.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,98 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test; + +import test.command.HelloQuitTest; +import test.command.ArticleTest; +import java.util.LinkedList; +import java.util.List; +import test.command.CapabilitiesTest; +import test.command.GroupTest; +import test.command.ListGroupTests; +import test.command.ListTest; +import test.command.NewGroupsTest; +import test.command.NextTest; +import test.command.OverTest; +import test.command.PostTest; + +/** + * Run this class to perform a full test. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class TestBench +{ + + public static void main(String[] args) + { + System.out.println("TestBench for NNTP (RFC3799) based servers "); + if(args.length < 1) + { + System.out.println("Usage: TestBench [:port]"); + return; + } + + String[] hostport = args[0].split(":"); + String host = hostport[0]; + int port = 119; + if(hostport.length == 2) + { + port = Integer.parseInt(hostport[1]); + } + + List tests = new LinkedList(); + + // Add tests to perform + tests.add(new HelloQuitTest()); + tests.add(new PostTest()); // Post before Article + tests.add(new ArticleTest()); + tests.add(new CapabilitiesTest()); + tests.add(new GroupTest()); + tests.add(new ListGroupTests()); + tests.add(new ListTest()); + tests.add(new NewGroupsTest()); + tests.add(new NextTest()); + tests.add(new OverTest()); + + // Perform all tests + for(AbstractTest test : tests) + { + try + { + test.connect(host, port); + int result = test.runTest(); + System.out.print(test.getClass().getName() + " finished with exit code " + result + "\t => "); + if(result == 0) + { + System.out.println("SUCCESS"); + } + else + { + System.out.println("FAILURE"); + } + } + catch(Exception ex) + { + System.out.println("Test " + test.getClass().getName() + " failed: " + ex.getLocalizedMessage()); + ex.printStackTrace(); + } + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/ArticleTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/ArticleTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,90 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Tests the ARTICLE command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ArticleTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + String line = readln(); + if(!line.startsWith("200 ")) + { + return 1; + } + + // Select a group (we assume that local.test is existing) + println("GROUP local.test"); + line = readln(); + if(!line.startsWith("211 ")) + { + println("GROUP test"); + line = readln(); + if(!line.startsWith("211 ")) + { + return 3; + } + } + + // Retrieve the first article + println("ARTICLE " + line.split(" ")[2]); + line = readln(); + if(!line.startsWith("220 ")) + { + return 4; + } + + while(!line.equals(".")) + { + line = readln(); + } + + // Retrieve currently selected article (without a parameter number!) + println("ARTICLE"); + line = readln(); + if(!line.startsWith("220 ")) + { + return 5; + } + + while(!line.equals(".")) + { + line = readln(); + } + + println("QUIT"); + line = readln(); + if(!line.startsWith("205 ")) + { + return 2; + } + + return 0; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/CapabilitiesTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/CapabilitiesTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,63 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Tests the CAPABILITIES command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class CapabilitiesTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + String line = readln(); + if(!line.startsWith("200 ")) + { + return 1; + } + + println("CAPABILITIES"); + line = readln(); + if(!line.startsWith("101")) + { + return 3; + } + + while(!line.equals(".")) + { + line = readln(); + } + + println("QUIT"); + line = readln(); + if(!line.startsWith("205 ")) + { + return 2; + } + + return 0; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/GroupTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/GroupTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,49 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Tests the GROUP command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class GroupTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + String line = readln(); + + println("GROUP misc.test"); + line = readln(); + if(line.startsWith("211 ")) + { + return 0; + } + else + { + return 1; + } + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/HelloQuitTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/HelloQuitTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,47 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Test: connects to server, waits for initial hello and quits. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class HelloQuitTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + String line = readln(); + if(!line.startsWith("200 ")) + return 1; + + println("QUIT"); + line = readln(); + if(!line.startsWith("205 ")) + return 2; + + return 0; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/ListGroupTests.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/ListGroupTests.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,38 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Tests the LISTGROUP command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ListGroupTests extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + return 1; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/ListTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/ListTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,49 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Blackbox Test testing the LIST command + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ListTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + println("LIST OVERVIEW.FMT"); + String line = readln(); + if(line.startsWith("215 ")) + { + while(!line.equals(".")) + { + line = readln(); + } + return 0; + } + else + return 1; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/NewGroupsTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/NewGroupsTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,37 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +package test.command; + +import test.AbstractTest; + +/** + * Tests the NEWGROUPS command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class NewGroupsTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + return 1; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/NextTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/NextTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,38 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Tests the NEXT command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class NextTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + return 1; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/OverTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/OverTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,135 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.List; +import test.AbstractTest; + +/** + * Tests the OVER/XOVER command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class OverTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + // Send HELLO to server + String line = readln(); + if(!line.startsWith("200 ")) + { + return 1; + } + + // Determine available groups + println("LIST"); + line = readln(); + if(!line.startsWith("215 ")) + { + return 2; + } + + List groups = new ArrayList(); + line = readln(); + for(;;) + { + if(line.equals(".")) + { + break; + } + else + { + groups.add(line); + line = readln(); + } + } + + if(groups.size() <= 0) + { + return 3; + } + + // Test OVER command on every group + for(String group : groups) + { + String groupName = group.split(" ")[0]; + println("GROUP " + groupName); + line = readln(); + if(!line.startsWith("211 ")) + { + return 4; + } + + String[] lineToks = line.split(" "); + println("XOVER " + lineToks[2] + "-" + lineToks[3]); + line = readln(); + if(line.startsWith("423")) + { + continue; + } + else if(!line.startsWith("224 ")) + { + return 5; + } + + line = readln(); + for(;;) + { + if(line == null) + { + return 7; + } + else if(line.equals(".")) + { + break; + } + + // Validate the line + lineToks = line.split("\t"); + if(lineToks.length < 6) + { + return 6; + } + else + { + Integer.parseInt(lineToks[0]); + + //SimpleDateFormat sdf = new SimpleDateFormat(group) + + if(!lineToks[4].startsWith("<") && !lineToks[4].endsWith(">")) + { + log.println("Invalid Message-ID: " + lineToks[1]); + log.flush(); + return 8; + } + } + + line = readln(); + } + } + + return 0; + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/PostTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/PostTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,82 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.command; + +import test.AbstractTest; + +/** + * Tests the POST command. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class PostTest extends AbstractTest +{ + + @Override + public int runTest() + throws Exception + { + String line = readln(); + if(!line.startsWith("200 ")) + { + return 1; + } + + println("POST"); + line = readln(); + if(!line.startsWith("340 ")) + { + return 1; + } + + // Post a sample article + postArticle("local.test"); + line = readln(); + if(line.startsWith("441 ")) + { + println("POST"); + line = readln(); + if(!line.startsWith("340 ")) + { + return 2; + } + + postArticle("test"); + line = readln(); + } + + if(!line.startsWith("240 ")) + { + return 3; + } + + return 0; + } + + private void postArticle(String toGroup) + { + println("Subject: A simple test mail"); + println("From: NNTP TestBench "); + println("Newsgroups: " + toGroup); + println(""); + println("Hello World!"); + println("."); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/command/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/command/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains conformance tests for various NNTP commands. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 test/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Root package for test classes. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/DaemonTests.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/DaemonTests.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,38 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.unit; + +import junit.textui.TestRunner; +import test.unit.daemon.NNTPConnectionTest; + +/** + * Tests the org.sonews.daemon package classes. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class DaemonTests +{ + + public static void main(String[] args) + { + System.out.println("NNTPConnectionTest"); + TestRunner.run(NNTPConnectionTest.class); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/UtilTests.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/UtilTests.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,47 @@ +/* + * StarOffice News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.unit; + +import junit.textui.TestRunner; +import test.unit.util.ResourceTest; +import test.unit.util.StringTemplateTest; +import test.unit.util.TimeoutMapTest; + + +/** + * Tests classes of package org.sonews.util. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class UtilTests +{ + + public static void main(String[] args) + { + System.out.println("StringTemplateTest"); + TestRunner.run(StringTemplateTest.class); + + System.out.println("TimeoutMapTest"); + TestRunner.run(TimeoutMapTest.class); + + System.out.println("ResourceTest"); + TestRunner.run(ResourceTest.class); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/daemon/NNTPConnectionTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/daemon/NNTPConnectionTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,122 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.unit.daemon; + +import java.io.IOException; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.nio.channels.SocketChannel; +import junit.framework.TestCase; +import org.sonews.daemon.NNTPConnection; +import org.sonews.daemon.command.ArticleCommand; +import org.sonews.daemon.command.CapabilitiesCommand; +import org.sonews.daemon.command.GroupCommand; +import org.sonews.daemon.command.UnsupportedCommand; + +/** + * Unit test for class NNTPConnection. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class NNTPConnectionTest extends TestCase +{ + + public void testLineReceived() + throws NoSuchMethodException, IllegalAccessException, InvocationTargetException + { + NNTPConnection conn = null; + + try + { + try + { + conn = new NNTPConnection(null); + fail("Should have raised an IllegalArgumentException"); + } + catch(IOException ex) {ex.printStackTrace();} + } + catch(IllegalArgumentException ex){} + + try + { + conn = new NNTPConnection(SocketChannel.open()); + } + catch(IOException ex) + { + ex.printStackTrace(); + } + + assertNotNull(conn); + + // Make interesting methods accessible + Class clazz = conn.getClass(); + Method methTryReadLock = clazz.getDeclaredMethod("tryReadLock", null); + methTryReadLock.setAccessible(true); + Method methLineReceived = clazz.getDeclaredMethod("lineReceived", new byte[0].getClass()); + methLineReceived.setAccessible(true); + + try + { + // conn.lineReceived(null); + methLineReceived.invoke(conn, null); + fail("Should have raised an IllegalArgumentException"); + } + catch(IllegalArgumentException ex){} + + try + { + // conn.lineReceived(new byte[0]); + methLineReceived.invoke(conn, new byte[0]); + fail("Should have raised IllegalStateException"); + } + catch(InvocationTargetException ex){} + + boolean tryReadLock = (Boolean)methTryReadLock.invoke(conn, null); + assertTrue(tryReadLock); + + // conn.lineReceived("MODE READER".getBytes()); + methLineReceived.invoke(conn, "MODE READER".getBytes()); + + // conn.lineReceived("sdkfsdjnfksjfdng ksdf gksjdfngk nskfng ksndfg ".getBytes()); + methLineReceived.invoke(conn, "sdkfsdjnfksjfdng ksdf ksndfg ".getBytes()); + + // conn.lineReceived(new byte[1024]); // Too long + methLineReceived.invoke(conn, new byte[1024]); + + Method mpcmdl = conn.getClass().getDeclaredMethod("parseCommandLine", String.class); + mpcmdl.setAccessible(true); + + Object result = mpcmdl.invoke(conn, ""); + assertNotNull(result); + assertTrue(result instanceof UnsupportedCommand); + + result = mpcmdl.invoke(conn, "aRtiCle"); + assertNotNull(result); + assertTrue(result instanceof ArticleCommand); + + result = mpcmdl.invoke(conn, "capAbilItIEs"); + assertNotNull(result); + assertTrue(result instanceof CapabilitiesCommand); + + result = mpcmdl.invoke(conn, "grOUp"); + assertNotNull(result); + assertTrue(result instanceof GroupCommand); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/daemon/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/daemon/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains Unittests for the org.sonews.daemon package. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Root package for Unittest classes. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/util/ResourceTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/util/ResourceTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,80 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.unit.util; + +import junit.framework.TestCase; +import org.sonews.util.io.Resource; + +/** + * Unit test for class org.sonews.util.io.Resource. + * @author Christian Lins + * @see org.sonews.util.io.Resource + * @since sonews/0.5.0 + */ +public class ResourceTest extends TestCase +{ + + public void testGetAsURL() + { + Object url; + + url = Resource.getAsURL(null); + assertNull(url); + + url = Resource.getAsURL("this is absolutely bullshit"); + assertNull(url); + + // This file should exist + url = Resource.getAsURL("org/sonews/daemon/Main.class"); + assertNotNull(url); + } + + public void testGetAsStream() + { + Object stream; + + stream = Resource.getAsStream(null); + assertNull(stream); + + stream = Resource.getAsStream("this is bullshit"); + assertNull(stream); + + stream = Resource.getAsStream("org/sonews/daemon/Main.class"); + assertNotNull(stream); + } + + public void testGetAsString() + { + String str; + + str = Resource.getAsString(null, true); + assertNull(str); + + str = Resource.getAsString("this is bullshit", true); + assertNull(str); + + str = Resource.getAsString("org/sonews/daemon/Main.class", true); + assertNotNull(str); + + str = Resource.getAsString("org/sonews/daemon/Main.class", false); + assertNotNull(str); + assertEquals(str.indexOf("\n"), -1); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/util/StringTemplateTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/util/StringTemplateTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,91 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.unit.util; + +import junit.framework.TestCase; +import org.sonews.util.StringTemplate; + +/** + * Unit test for class org.sonews.util.StringTemplate. + * @author Christian Lins + * @see org.sonews.util.StringTemplate + * @since sonews/0.5.0 + */ +public class StringTemplateTest extends TestCase +{ + + private static final String template = "Hello %WORLD and others!"; + + public StringTemplateTest() + { + super("StringTemplateTest"); + } + + public void testCtor() + { + StringTemplate st; + + try + { + st = new StringTemplate(null); + fail("Should have raised an IllegalArgumentException"); + } + catch(IllegalArgumentException ex) {} + + st = new StringTemplate(template); + assertNotNull(st); + + try + { + st = new StringTemplate(template, null); + fail("Should have raised an IllegalArgumentException"); + } + catch(IllegalArgumentException ex) {} + + st = new StringTemplate(template, "?"); + assertNotNull(st); + + st = new StringTemplate(template, "%"); + assertNotNull(st); + } + + public void testSetter() + { + StringTemplate st = new StringTemplate(template); + + try + { + st.set("WORLD", null); + fail("Should have raised an IllegalArgumentException"); + } + catch(IllegalArgumentException ex){} + + st.set("WORLD", "Universe"); + } + + public void testToString() + { + StringTemplate st = new StringTemplate(template); + st.set("WORLD", "Universe"); + + String result = st.toString(); + assertNotNull(result); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/util/TimeoutMapTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/util/TimeoutMapTest.java Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,70 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package test.unit.util; + +import junit.framework.TestCase; +import org.sonews.util.TimeoutMap; + +/** + * Unit test for class org.sonews.util.TimeoutMap. + * @author Christian Lins + * @since sonews/0.5.0 + * @see org.sonews.util.TimeoutMap + */ +public class TimeoutMapTest extends TestCase +{ + + public TimeoutMapTest() + { + super("TimeoutMapTest"); + } + + public void testTimeoutBehaviour() + { + TimeoutMap tm = new TimeoutMap(1000); + Object testobj = new Object(); + + tm.put("testkey", testobj); + + assertNotNull(tm.get("testkey")); + assertTrue(tm.containsKey("testkey")); + assertTrue(tm.contains(testobj)); + + try + { + Thread.sleep(800); + } + catch(InterruptedException ex) { ex.printStackTrace(); } + + assertNotNull(tm.get("testkey")); + assertTrue(tm.containsKey("testkey")); + assertTrue(tm.contains(testobj)); + + try + { + Thread.sleep(200); + } + catch(InterruptedException ex) { ex.printStackTrace(); } + + assertNull(tm.get("testkey")); + assertFalse(tm.containsKey("testkey")); + assertFalse(tm.contains(testobj)); + } + +} diff -r f907866f0e4b -r 6fceb66e1ad7 test/unit/util/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/unit/util/package.html Fri Jun 26 16:48:50 2009 +0200 @@ -0,0 +1,1 @@ +Contains Unittests for the package org.sonews.util. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/AUTHORS --- a/trunk/AUTHORS Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,26 +0,0 @@ -AUTHORS & CREDITS -================= - -As most software applications "StarOffice News Server" is based on the work -of individuals or projects. These fine people contributing to the OpenSource -community are mentioned here: - -StarOffice News Server ----------------------- -(c)Copyright 2009 by - * Sun Microsystems, Inc. - * Christian Lins - -based partly upon - -Neat NNTP Daemon (n3tpd) ------------------------- -(c)Copyright 2007, 2008 by Christian Lins - -based partly upon - -tnntpd ------- -(c)Copyright 2003 by Dennis Schwerdel - -If you find someone missing here, please contact the project leader! \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/COPYING --- a/trunk/COPYING Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,674 +0,0 @@ - GNU GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The GNU General Public License is a free, copyleft license for -software and other kinds of works. - - The licenses for most software and other practical works are designed -to take away your freedom to share and change the works. By contrast, -the GNU General Public License is intended to guarantee your freedom to -share and change all versions of a program--to make sure it remains free -software for all its users. We, the Free Software Foundation, use the -GNU General Public License for most of our software; it applies also to -any other work released this way by its authors. You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -them if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs, and that you know you can do these things. - - To protect your rights, we need to prevent others from denying you -these rights or asking you to surrender the rights. Therefore, you have -certain responsibilities if you distribute copies of the software, or if -you modify it: responsibilities to respect the freedom of others. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must pass on to the recipients the same -freedoms that you received. You must make sure that they, too, receive -or can get the source code. And you must show them these terms so they -know their rights. - - Developers that use the GNU GPL protect your rights with two steps: -(1) assert copyright on the software, and (2) offer you this License -giving you legal permission to copy, distribute and/or modify it. - - For the developers' and authors' protection, the GPL clearly explains -that there is no warranty for this free software. For both users' and -authors' sake, the GPL requires that modified versions be marked as -changed, so that their problems will not be attributed erroneously to -authors of previous versions. - - Some devices are designed to deny users access to install or run -modified versions of the software inside them, although the manufacturer -can do so. This is fundamentally incompatible with the aim of -protecting users' freedom to change the software. The systematic -pattern of such abuse occurs in the area of products for individuals to -use, which is precisely where it is most unacceptable. Therefore, we -have designed this version of the GPL to prohibit the practice for those -products. If such problems arise substantially in other domains, we -stand ready to extend this provision to those domains in future versions -of the GPL, as needed to protect the freedom of users. - - Finally, every program is threatened constantly by software patents. -States should not allow patents to restrict development and use of -software on general-purpose computers, but in those that do, we wish to -avoid the special danger that patents applied to a free program could -make it effectively proprietary. To prevent this, the GPL assures that -patents cannot be used to render the program non-free. - - The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - - 0. Definitions. - - "This License" refers to version 3 of the GNU General Public License. - - "Copyright" also means copyright-like laws that apply to other kinds of -works, such as semiconductor masks. - - "The Program" refers to any copyrightable work licensed under this -License. Each licensee is addressed as "you". "Licensees" and -"recipients" may be individuals or organizations. - - To "modify" a work means to copy from or adapt all or part of the work -in a fashion requiring copyright permission, other than the making of an -exact copy. The resulting work is called a "modified version" of the -earlier work or a work "based on" the earlier work. - - A "covered work" means either the unmodified Program or a work based -on the Program. - - To "propagate" a work means to do anything with it that, without -permission, would make you directly or secondarily liable for -infringement under applicable copyright law, except executing it on a -computer or modifying a private copy. Propagation includes copying, -distribution (with or without modification), making available to the -public, and in some countries other activities as well. - - To "convey" a work means any kind of propagation that enables other -parties to make or receive copies. Mere interaction with a user through -a computer network, with no transfer of a copy, is not conveying. - - An interactive user interface displays "Appropriate Legal Notices" -to the extent that it includes a convenient and prominently visible -feature that (1) displays an appropriate copyright notice, and (2) -tells the user that there is no warranty for the work (except to the -extent that warranties are provided), that licensees may convey the -work under this License, and how to view a copy of this License. If -the interface presents a list of user commands or options, such as a -menu, a prominent item in the list meets this criterion. - - 1. Source Code. - - The "source code" for a work means the preferred form of the work -for making modifications to it. "Object code" means any non-source -form of a work. - - A "Standard Interface" means an interface that either is an official -standard defined by a recognized standards body, or, in the case of -interfaces specified for a particular programming language, one that -is widely used among developers working in that language. - - The "System Libraries" of an executable work include anything, other -than the work as a whole, that (a) is included in the normal form of -packaging a Major Component, but which is not part of that Major -Component, and (b) serves only to enable use of the work with that -Major Component, or to implement a Standard Interface for which an -implementation is available to the public in source code form. A -"Major Component", in this context, means a major essential component -(kernel, window system, and so on) of the specific operating system -(if any) on which the executable work runs, or a compiler used to -produce the work, or an object code interpreter used to run it. - - The "Corresponding Source" for a work in object code form means all -the source code needed to generate, install, and (for an executable -work) run the object code and to modify the work, including scripts to -control those activities. However, it does not include the work's -System Libraries, or general-purpose tools or generally available free -programs which are used unmodified in performing those activities but -which are not part of the work. For example, Corresponding Source -includes interface definition files associated with source files for -the work, and the source code for shared libraries and dynamically -linked subprograms that the work is specifically designed to require, -such as by intimate data communication or control flow between those -subprograms and other parts of the work. - - The Corresponding Source need not include anything that users -can regenerate automatically from other parts of the Corresponding -Source. - - The Corresponding Source for a work in source code form is that -same work. - - 2. Basic Permissions. - - All rights granted under this License are granted for the term of -copyright on the Program, and are irrevocable provided the stated -conditions are met. This License explicitly affirms your unlimited -permission to run the unmodified Program. The output from running a -covered work is covered by this License only if the output, given its -content, constitutes a covered work. This License acknowledges your -rights of fair use or other equivalent, as provided by copyright law. - - You may make, run and propagate covered works that you do not -convey, without conditions so long as your license otherwise remains -in force. You may convey covered works to others for the sole purpose -of having them make modifications exclusively for you, or provide you -with facilities for running those works, provided that you comply with -the terms of this License in conveying all material for which you do -not control copyright. Those thus making or running the covered works -for you must do so exclusively on your behalf, under your direction -and control, on terms that prohibit them from making any copies of -your copyrighted material outside their relationship with you. - - Conveying under any other circumstances is permitted solely under -the conditions stated below. Sublicensing is not allowed; section 10 -makes it unnecessary. - - 3. Protecting Users' Legal Rights From Anti-Circumvention Law. - - No covered work shall be deemed part of an effective technological -measure under any applicable law fulfilling obligations under article -11 of the WIPO copyright treaty adopted on 20 December 1996, or -similar laws prohibiting or restricting circumvention of such -measures. - - When you convey a covered work, you waive any legal power to forbid -circumvention of technological measures to the extent such circumvention -is effected by exercising rights under this License with respect to -the covered work, and you disclaim any intention to limit operation or -modification of the work as a means of enforcing, against the work's -users, your or third parties' legal rights to forbid circumvention of -technological measures. - - 4. Conveying Verbatim Copies. - - You may convey verbatim copies of the Program's source code as you -receive it, in any medium, provided that you conspicuously and -appropriately publish on each copy an appropriate copyright notice; -keep intact all notices stating that this License and any -non-permissive terms added in accord with section 7 apply to the code; -keep intact all notices of the absence of any warranty; and give all -recipients a copy of this License along with the Program. - - You may charge any price or no price for each copy that you convey, -and you may offer support or warranty protection for a fee. - - 5. Conveying Modified Source Versions. - - You may convey a work based on the Program, or the modifications to -produce it from the Program, in the form of source code under the -terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified - it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is - released under this License and any conditions added under section - 7. This requirement modifies the requirement in section 4 to - "keep intact all notices". - - c) You must license the entire work, as a whole, under this - License to anyone who comes into possession of a copy. This - License will therefore apply, along with any applicable section 7 - additional terms, to the whole of the work, and all its parts, - regardless of how they are packaged. This License gives no - permission to license the work in any other way, but it does not - invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display - Appropriate Legal Notices; however, if the Program has interactive - interfaces that do not display Appropriate Legal Notices, your - work need not make them do so. - - A compilation of a covered work with other separate and independent -works, which are not by their nature extensions of the covered work, -and which are not combined with it such as to form a larger program, -in or on a volume of a storage or distribution medium, is called an -"aggregate" if the compilation and its resulting copyright are not -used to limit the access or legal rights of the compilation's users -beyond what the individual works permit. Inclusion of a covered work -in an aggregate does not cause this License to apply to the other -parts of the aggregate. - - 6. Conveying Non-Source Forms. - - You may convey a covered work in object code form under the terms -of sections 4 and 5, provided that you also convey the -machine-readable Corresponding Source under the terms of this License, -in one of these ways: - - a) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by the - Corresponding Source fixed on a durable physical medium - customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by a - written offer, valid for at least three years and valid for as - long as you offer spare parts or customer support for that product - model, to give anyone who possesses the object code either (1) a - copy of the Corresponding Source for all the software in the - product that is covered by this License, on a durable physical - medium customarily used for software interchange, for a price no - more than your reasonable cost of physically performing this - conveying of source, or (2) access to copy the - Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the - written offer to provide the Corresponding Source. This - alternative is allowed only occasionally and noncommercially, and - only if you received the object code with such an offer, in accord - with subsection 6b. - - d) Convey the object code by offering access from a designated - place (gratis or for a charge), and offer equivalent access to the - Corresponding Source in the same way through the same place at no - further charge. You need not require recipients to copy the - Corresponding Source along with the object code. If the place to - copy the object code is a network server, the Corresponding Source - may be on a different server (operated by you or a third party) - that supports equivalent copying facilities, provided you maintain - clear directions next to the object code saying where to find the - Corresponding Source. Regardless of what server hosts the - Corresponding Source, you remain obligated to ensure that it is - available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided - you inform other peers where the object code and Corresponding - Source of the work are being offered to the general public at no - charge under subsection 6d. - - A separable portion of the object code, whose source code is excluded -from the Corresponding Source as a System Library, need not be -included in conveying the object code work. - - A "User Product" is either (1) a "consumer product", which means any -tangible personal property which is normally used for personal, family, -or household purposes, or (2) anything designed or sold for incorporation -into a dwelling. In determining whether a product is a consumer product, -doubtful cases shall be resolved in favor of coverage. For a particular -product received by a particular user, "normally used" refers to a -typical or common use of that class of product, regardless of the status -of the particular user or of the way in which the particular user -actually uses, or expects or is expected to use, the product. A product -is a consumer product regardless of whether the product has substantial -commercial, industrial or non-consumer uses, unless such uses represent -the only significant mode of use of the product. - - "Installation Information" for a User Product means any methods, -procedures, authorization keys, or other information required to install -and execute modified versions of a covered work in that User Product from -a modified version of its Corresponding Source. The information must -suffice to ensure that the continued functioning of the modified object -code is in no case prevented or interfered with solely because -modification has been made. - - If you convey an object code work under this section in, or with, or -specifically for use in, a User Product, and the conveying occurs as -part of a transaction in which the right of possession and use of the -User Product is transferred to the recipient in perpetuity or for a -fixed term (regardless of how the transaction is characterized), the -Corresponding Source conveyed under this section must be accompanied -by the Installation Information. But this requirement does not apply -if neither you nor any third party retains the ability to install -modified object code on the User Product (for example, the work has -been installed in ROM). - - The requirement to provide Installation Information does not include a -requirement to continue to provide support service, warranty, or updates -for a work that has been modified or installed by the recipient, or for -the User Product in which it has been modified or installed. Access to a -network may be denied when the modification itself materially and -adversely affects the operation of the network or violates the rules and -protocols for communication across the network. - - Corresponding Source conveyed, and Installation Information provided, -in accord with this section must be in a format that is publicly -documented (and with an implementation available to the public in -source code form), and must require no special password or key for -unpacking, reading or copying. - - 7. Additional Terms. - - "Additional permissions" are terms that supplement the terms of this -License by making exceptions from one or more of its conditions. -Additional permissions that are applicable to the entire Program shall -be treated as though they were included in this License, to the extent -that they are valid under applicable law. If additional permissions -apply only to part of the Program, that part may be used separately -under those permissions, but the entire Program remains governed by -this License without regard to the additional permissions. - - When you convey a copy of a covered work, you may at your option -remove any additional permissions from that copy, or from any part of -it. (Additional permissions may be written to require their own -removal in certain cases when you modify the work.) You may place -additional permissions on material, added by you to a covered work, -for which you have or can give appropriate copyright permission. - - Notwithstanding any other provision of this License, for material you -add to a covered work, you may (if authorized by the copyright holders of -that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the - terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or - author attributions in that material or in the Appropriate Legal - Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or - requiring that modified versions of such material be marked in - reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or - authors of the material; or - - e) Declining to grant rights under trademark law for use of some - trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that - material by anyone who conveys the material (or modified versions of - it) with contractual assumptions of liability to the recipient, for - any liability that these contractual assumptions directly impose on - those licensors and authors. - - All other non-permissive additional terms are considered "further -restrictions" within the meaning of section 10. If the Program as you -received it, or any part of it, contains a notice stating that it is -governed by this License along with a term that is a further -restriction, you may remove that term. If a license document contains -a further restriction but permits relicensing or conveying under this -License, you may add to a covered work material governed by the terms -of that license document, provided that the further restriction does -not survive such relicensing or conveying. - - If you add terms to a covered work in accord with this section, you -must place, in the relevant source files, a statement of the -additional terms that apply to those files, or a notice indicating -where to find the applicable terms. - - Additional terms, permissive or non-permissive, may be stated in the -form of a separately written license, or stated as exceptions; -the above requirements apply either way. - - 8. Termination. - - You may not propagate or modify a covered work except as expressly -provided under this License. Any attempt otherwise to propagate or -modify it is void, and will automatically terminate your rights under -this License (including any patent licenses granted under the third -paragraph of section 11). - - However, if you cease all violation of this License, then your -license from a particular copyright holder is reinstated (a) -provisionally, unless and until the copyright holder explicitly and -finally terminates your license, and (b) permanently, if the copyright -holder fails to notify you of the violation by some reasonable means -prior to 60 days after the cessation. - - Moreover, your license from a particular copyright holder is -reinstated permanently if the copyright holder notifies you of the -violation by some reasonable means, this is the first time you have -received notice of violation of this License (for any work) from that -copyright holder, and you cure the violation prior to 30 days after -your receipt of the notice. - - Termination of your rights under this section does not terminate the -licenses of parties who have received copies or rights from you under -this License. If your rights have been terminated and not permanently -reinstated, you do not qualify to receive new licenses for the same -material under section 10. - - 9. Acceptance Not Required for Having Copies. - - You are not required to accept this License in order to receive or -run a copy of the Program. Ancillary propagation of a covered work -occurring solely as a consequence of using peer-to-peer transmission -to receive a copy likewise does not require acceptance. However, -nothing other than this License grants you permission to propagate or -modify any covered work. These actions infringe copyright if you do -not accept this License. Therefore, by modifying or propagating a -covered work, you indicate your acceptance of this License to do so. - - 10. Automatic Licensing of Downstream Recipients. - - Each time you convey a covered work, the recipient automatically -receives a license from the original licensors, to run, modify and -propagate that work, subject to this License. You are not responsible -for enforcing compliance by third parties with this License. - - An "entity transaction" is a transaction transferring control of an -organization, or substantially all assets of one, or subdividing an -organization, or merging organizations. If propagation of a covered -work results from an entity transaction, each party to that -transaction who receives a copy of the work also receives whatever -licenses to the work the party's predecessor in interest had or could -give under the previous paragraph, plus a right to possession of the -Corresponding Source of the work from the predecessor in interest, if -the predecessor has it or can get it with reasonable efforts. - - You may not impose any further restrictions on the exercise of the -rights granted or affirmed under this License. For example, you may -not impose a license fee, royalty, or other charge for exercise of -rights granted under this License, and you may not initiate litigation -(including a cross-claim or counterclaim in a lawsuit) alleging that -any patent claim is infringed by making, using, selling, offering for -sale, or importing the Program or any portion of it. - - 11. Patents. - - A "contributor" is a copyright holder who authorizes use under this -License of the Program or a work on which the Program is based. The -work thus licensed is called the contributor's "contributor version". - - A contributor's "essential patent claims" are all patent claims -owned or controlled by the contributor, whether already acquired or -hereafter acquired, that would be infringed by some manner, permitted -by this License, of making, using, or selling its contributor version, -but do not include claims that would be infringed only as a -consequence of further modification of the contributor version. For -purposes of this definition, "control" includes the right to grant -patent sublicenses in a manner consistent with the requirements of -this License. - - Each contributor grants you a non-exclusive, worldwide, royalty-free -patent license under the contributor's essential patent claims, to -make, use, sell, offer for sale, import and otherwise run, modify and -propagate the contents of its contributor version. - - In the following three paragraphs, a "patent license" is any express -agreement or commitment, however denominated, not to enforce a patent -(such as an express permission to practice a patent or covenant not to -sue for patent infringement). To "grant" such a patent license to a -party means to make such an agreement or commitment not to enforce a -patent against the party. - - If you convey a covered work, knowingly relying on a patent license, -and the Corresponding Source of the work is not available for anyone -to copy, free of charge and under the terms of this License, through a -publicly available network server or other readily accessible means, -then you must either (1) cause the Corresponding Source to be so -available, or (2) arrange to deprive yourself of the benefit of the -patent license for this particular work, or (3) arrange, in a manner -consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have -actual knowledge that, but for the patent license, your conveying the -covered work in a country, or your recipient's use of the covered work -in a country, would infringe one or more identifiable patents in that -country that you have reason to believe are valid. - - If, pursuant to or in connection with a single transaction or -arrangement, you convey, or propagate by procuring conveyance of, a -covered work, and grant a patent license to some of the parties -receiving the covered work authorizing them to use, propagate, modify -or convey a specific copy of the covered work, then the patent license -you grant is automatically extended to all recipients of the covered -work and works based on it. - - A patent license is "discriminatory" if it does not include within -the scope of its coverage, prohibits the exercise of, or is -conditioned on the non-exercise of one or more of the rights that are -specifically granted under this License. You may not convey a covered -work if you are a party to an arrangement with a third party that is -in the business of distributing software, under which you make payment -to the third party based on the extent of your activity of conveying -the work, and under which the third party grants, to any of the -parties who would receive the covered work from you, a discriminatory -patent license (a) in connection with copies of the covered work -conveyed by you (or copies made from those copies), or (b) primarily -for and in connection with specific products or compilations that -contain the covered work, unless you entered into that arrangement, -or that patent license was granted, prior to 28 March 2007. - - Nothing in this License shall be construed as excluding or limiting -any implied license or other defenses to infringement that may -otherwise be available to you under applicable patent law. - - 12. No Surrender of Others' Freedom. - - If conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot convey a -covered work so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you -to collect a royalty for further conveying from those to whom you convey -the Program, the only way you could satisfy both those terms and this -License would be to refrain entirely from conveying the Program. - - 13. Use with the GNU Affero General Public License. - - Notwithstanding any other provision of this License, you have -permission to link or combine any covered work with a work licensed -under version 3 of the GNU Affero General Public License into a single -combined work, and to convey the resulting work. The terms of this -License will continue to apply to the part which is the covered work, -but the special requirements of the GNU Affero General Public License, -section 13, concerning interaction through a network will apply to the -combination as such. - - 14. Revised Versions of this License. - - The Free Software Foundation may publish revised and/or new versions of -the GNU General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - - Each version is given a distinguishing version number. If the -Program specifies that a certain numbered version of the GNU General -Public License "or any later version" applies to it, you have the -option of following the terms and conditions either of that numbered -version or of any later version published by the Free Software -Foundation. If the Program does not specify a version number of the -GNU General Public License, you may choose any version ever published -by the Free Software Foundation. - - If the Program specifies that a proxy can decide which future -versions of the GNU General Public License can be used, that proxy's -public statement of acceptance of a version permanently authorizes you -to choose that version for the Program. - - Later license versions may give you additional or different -permissions. However, no additional obligations are imposed on any -author or copyright holder as a result of your choosing to follow a -later version. - - 15. Disclaimer of Warranty. - - THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY -APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT -HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY -OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, -THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM -IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF -ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - - 16. Limitation of Liability. - - IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS -THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY -GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE -USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF -DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD -PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), -EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF -SUCH DAMAGES. - - 17. Interpretation of Sections 15 and 16. - - If the disclaimer of warranty and limitation of liability provided -above cannot be given local legal effect according to their terms, -reviewing courts shall apply local law that most closely approximates -an absolute waiver of all civil liability in connection with the -Program, unless a warranty or assumption of liability accompanies a -copy of the Program in return for a fee. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -state the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - - If the program does terminal interaction, make it output a short -notice like this when it starts in an interactive mode: - - Copyright (C) - This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, your program's commands -might be different; for a GUI interface, you would use an "about box". - - You should also get your employer (if you work as a programmer) or school, -if any, to sign a "copyright disclaimer" for the program, if necessary. -For more information on this, and how to apply and follow the GNU GPL, see -. - - The GNU General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications with -the library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License. But first, please read -. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/README --- a/trunk/README Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,24 +0,0 @@ -n3tpd README -============ - -Prerequisites: --------------- - -* Java Runtime, Version 1.5 -* MySQL, Version 5 or higher -* Java-MySQL-Connector (JDBC driver) - - -Installation: -------------- - -* Create a database in your DBMS, e.g. named like 'n3tpd_data' -* Create the necessary table structure using the helpers/table.sql file -* Customize the settings within the n3tpd.conf file -* Invoke 'java -jar n3tpd.jar' to start the daemon - -Bugs and other Issues: ----------------------- - -Please mail them to christian.lins@web.de or, better, issue them -into our bugtracker at http://bugs.netvader.net/ . \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/TODO --- a/trunk/TODO Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,10 +0,0 @@ -TODO -==== - -* Implement new SQL table structure -* Implement a --init-tables command that creates the necessary database/tables - -ROADMAP -======= - -Version 1.0 must be fully RFC3977 compliant. \ No newline at end of file diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/Config.java --- a/trunk/com/so/news/Config.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,198 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news; - -import java.io.FileInputStream; -import java.io.FileNotFoundException; -import java.io.FileOutputStream; -import java.io.IOException; -import java.util.Properties; - -/** - * Manages the n3tpd configuration. - * @author Christian Lins - */ -public class Config -{ - /** The filename of the logfile */ - public static final String CONFIG_N3TPD_LOGFILE = "n3tpd.logfile"; - - /** The filename of the config file that is loaded on startup */ - public static final String FILE = "n3tpd.conf"; - - private static final Properties defaultConfig = new Properties(); - - private static Config instance = null; - - static - { - // Set some default values - defaultConfig.setProperty("n3tpd.article.lifetime", "300"); // 300 days - defaultConfig.setProperty("n3tpd.article.maxsize", "100"); // 100 kbyte - defaultConfig.setProperty("n3tpd.port", "119"); - defaultConfig.setProperty("n3tpd.auxport", "8080"); - defaultConfig.setProperty("n3tpd.server.backlog", "10"); - defaultConfig.setProperty("n3tpd.hostname", "localhost"); - defaultConfig.setProperty("n3tpd.storage.database", "jdbc:mysql://localhost/n3tpd_data"); - defaultConfig.setProperty("n3tpd.storage.dbmsdriver", "com.mysql.jdbc.Driver"); - defaultConfig.setProperty("n3tpd.storage.user", "n3tpd_user"); - defaultConfig.setProperty("n3tpd.storage.password", "mysecret"); - - instance = new Config(); - } - - /** - * @return A Config instance - */ - public static Config getInstance() - { - return instance; - } - - // Every config instance is initialized with the default values. - private Properties settings = (Properties)defaultConfig.clone(); - - /** - * Config is a singelton class with only one instance at time. - * So the constructor is private to prevent the creation of more - * then one Config instance. - * @see Config.getInstance() to retrieve an instance of Config - */ - private Config() - { - try - { - // Load settings from file - load(); - } - catch(IOException e) - { - e.printStackTrace(); - } - } - - /** - * Loads the configuration from the config file. By default this is done - * by the (private) constructor but it can be useful to reload the config - * by invoking this method. - * @throws IOException - */ - public void load() throws IOException - { - try - { - settings.load(new FileInputStream(FILE)); - } - catch (FileNotFoundException e) - { - save(); - } - } - - /** - * Saves this Config to the config file. By default this is done - * at program end. - * @throws FileNotFoundException - * @throws IOException - */ - public void save() throws FileNotFoundException, IOException - { - settings.store(new FileOutputStream(FILE), "N3TPD Config File"); - } - - /** - * Returns the value that is stored within this config - * identified by the given key. If the key cannot be found - * the default value is returned. - * @param key Key to identify the value. - * @param def The default value that is returned if the key - * is not found in this Config. - * @return - */ - public String get(String key, String def) - { - return settings.getProperty(key, def); - } - - /** - * Returns the value that is stored within this config - * identified by the given key. If the key cannot be found - * the default value is returned. - * @param key Key to identify the value. - * @param def The default value that is returned if the key - * is not found in this Config. - * @return - */ - public int get(String key, int def) - { - try - { - String val = get(key); - return Integer.parseInt(val); - } - catch(Exception e) - { - return def; - } - } - - /** - * Returns the value that is stored within this config - * identified by the given key. If the key cannot be found - * the default value is returned. - * @param key Key to identify the value. - * @param def The default value that is returned if the key - * is not found in this Config. - * @return - */ - public long get(String key, long def) - { - try - { - String val = get(key); - return Long.parseLong(val); - } - catch(Exception e) - { - return def; - } - } - - /** - * Returns the value for the given key or null if the - * key is not found in this Config. - * @param key - * @return - */ - private String get(String key) - { - return settings.getProperty(key); - } - - /** - * Sets the value for a given key. - * @param key - * @param value - */ - public void set(String key, String value) - { - settings.setProperty(key, value); - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/Debug.java --- a/trunk/com/so/news/Debug.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,95 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news; - -import java.io.FileOutputStream; -import java.io.IOException; -import java.io.PrintStream; -import java.util.Date; - -/** - * Provides logging and debugging methods. - * @author Christian Lins - */ -public class Debug -{ - private static Debug instance = null; - - /** - * Returns the singelton instance of this class. - */ - public static Debug getInstance() - { - if(instance == null) - instance = new Debug(); - - return instance; - } - - private PrintStream out = System.err; - - /** - * This class is a singelton class. The constructor is private to prevent - * the creation of more than one instance. - */ - private Debug() - { - try - { - String filename = Config.getInstance().get(Config.CONFIG_N3TPD_LOGFILE, "n3tpd.log"); - - this.out = new PrintStream(new FileOutputStream(filename)); - } - catch(IOException e) - { - e.printStackTrace(); - } - } - - /** - * Returns the debug output PrintStream. By default this is System.err. - */ - public PrintStream getStream() - { - return out; - } - - /** - * Writes the given message to the debug output. - * @param msg A String message or an object. - */ - public void log(Object msg) - { - log(out, msg); - log(System.out, msg); - } - - /** - * Writes the given debug message to the given PrintStream. - * @param out - * @param msg - */ - public void log(PrintStream out, Object msg) - { - out.print(new Date().toString()); - out.print(": "); - out.println(msg.toString()); - out.flush(); - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/Main.java --- a/trunk/com/so/news/Main.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,96 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news; - -import java.net.BindException; - -import java.sql.Driver; -import java.sql.DriverManager; -import java.util.Enumeration; - -import com.so.news.storage.Database; -import com.so.news.storage.Purger; - -/** - * Startup class of the daemon. - * @author Christian Lins - */ -public class Main -{ - /** Version information of the StarOffice News daemon */ - public static final String VERSION = "StarOffice News Server 0.5alpha1"; - - /** - * The main entrypoint. - * @param args - * @throws Exception - */ - public static void main(String args[]) throws Exception - { - System.out.println(VERSION); - - // Command line arguments - boolean auxPort = false; - - for(int n = 0; n < args.length; n++) - { - if(args[n].equals("--dumpjdbcdriver")) - { - System.out.println("Available JDBC drivers:"); - Enumeration drvs = DriverManager.getDrivers(); - while(drvs.hasMoreElements()) - System.out.println(drvs.nextElement()); - return; - } - else if(args[n].equals("--useaux")) - auxPort = true; - } - - // Try to load the Database - try - { - Database.arise(); - } - catch(Exception ex) - { - ex.printStackTrace(Debug.getInstance().getStream()); - System.err.println("Database initialization failed with " + ex.toString()); - - return; - } - - // Start the n3tpd garbage collector - new Purger().start(); - - // Start the listening daemon - try - { - new NNTPDaemon(false).start(); - } - catch(BindException ex) - { - ex.printStackTrace(Debug.getInstance().getStream()); - System.err.println("Could not bind to interface. Perhaps you need superuser rights?"); - } - - // Start auxilary listening port... - if(auxPort) - new NNTPDaemon(true).start(); - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/NNTPConnection.java --- a/trunk/com/so/news/NNTPConnection.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,395 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news; - -import java.io.BufferedReader; -import java.io.BufferedWriter; -import java.io.File; -import java.io.InputStreamReader; -import java.io.IOException; -import java.io.OutputStreamWriter; -import java.net.Socket; -import java.net.SocketException; -import java.text.SimpleDateFormat; -import java.util.Date; -import java.util.LinkedList; -import java.util.List; -import com.so.news.command.ArticleCommand; -import com.so.news.command.GroupCommand; -import com.so.news.command.ListCommand; -import com.so.news.command.PostCommand; -import com.so.news.command.OverCommand; -import com.so.news.storage.Article; -import com.so.news.storage.Group; - -/** - * Represents the connection between the server and one client. - * @author Christian Lins (christian.lins@web.de) - */ -public class NNTPConnection extends Thread -{ - public static final String NEWLINE = "\r\n"; - public static final String MESSAGE_ID_PATTERN = "<[^>]+>"; - - private boolean debug - = Boolean.parseBoolean(Config.getInstance().get("n3tpd.debug", "false")); - private Socket socket; - private boolean exit = false; - private BufferedWriter out; - private BufferedReader in; - private Article currentArticle = null; - private Group currentGroup = null; - - /** - * Creates a new NNTPConnection instance using the given connected Socket. - * @param socket - * @throws java.io.IOException - */ - public NNTPConnection(Socket socket) - throws IOException - { - this.socket = socket; - this.in = new BufferedReader(new InputStreamReader(socket.getInputStream())); - this.out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())); - // TODO: The output stream should be of type PrintStream so that many - // of the printX() methods of this class can go to trash - - setDaemon(true); // Exits if the main thread is killed - } - - /** - * Closes the associated socket end exits the Thread. - */ - public void exit() - { - try - { - exit = true; - socket.close(); - } - catch (Exception e) - { - e.printStackTrace(); - } - } - - /** - * Prints a CharSequence to the sockets output stream. - */ - public void print(CharSequence s) throws IOException - { - out.append(s); - } - - public void println(CharSequence s) throws IOException - { - print(s); - print(NEWLINE); - if (debug) - System.out.println("<<< " + s); - } - - public void printStatus(int code, String description) throws IOException - { - println("" + code + " " + description); - flush(); - } - - public void printTextLine(CharSequence line) throws IOException - { - if (line.length() > 0 && line.charAt(0) == '.') - print(".."); - println(line); - } - - public void printTextPart(CharSequence text) throws IOException - { - String[] lines = text.toString().split(NEWLINE); - for (String line : lines) - printTextLine(line); - } - - public void printText(CharSequence text) throws IOException - { - printTextPart(text); - println("."); - flush(); - } - - public void flush() throws IOException - { - out.flush(); - } - - public String readln() throws IOException - { - String s = in.readLine(); - if (s == null) - throw new IOException("Socket closed"); - if (debug) - System.out.println(">>> " + s); - return s; - } - - public String[] readCommand() throws IOException - { - return readln().split("[ ]+"); - } - - public List readText() throws IOException - { - List l = new LinkedList(); - String s; - do - { - s = readln(); - if (!s.equals(".")) - { - if (s.startsWith("..")) - s = s.substring(1); - l.add(s); - } - } - while (!s.equals(".")); - return l; - } - - public String readTextLine() throws IOException - { - String s = null; - do - { - s = readln(); - } - while (s == null); - if (s.equals(".")) - return null; - if (s.startsWith("..")) - s = s.substring(1); - return s; - } - - public void setCurrentArticle(Article current) - { - currentArticle = current; - } - - public Article getCurrentArticle() - { - return currentArticle; - } - - public void setCurrentGroup(Group current) - { - currentGroup = current; - } - - public Group getCurrentGroup() - { - return currentGroup; - } - - private void processCommand(String[] command) - throws Exception - { - if (command.length == 0) - return; // TODO Error - - String commandName = command[0]; - - // RFC977 - // TODO HELP command - // TODO NEWGROUPS command - // TODO NEWNEWS command - - // RFC2980 - // TODO LIST ACTIVE command - // TODO LIST ACTIVE.TIMES command - // TODO LIST DISTRIBUTIONS command - // TODO LIST DISTRIB.PATS command - // TODO XGTITLE command - // TODO XHDR command - // TODO XPAT command - // TODO XPATH command - // TODO XROVER command - // TODO XTHREAD command - // TODO AUTHINFO command - - // STANDARD COMMANDS - if (commandName.equalsIgnoreCase("ARTICLE") - || commandName.equalsIgnoreCase("STAT") - || commandName.equalsIgnoreCase("HEAD") - || commandName.equalsIgnoreCase("BODY")) - { - ArticleCommand cmd = new ArticleCommand(this); - cmd.process(command); - } - - else if (commandName.equalsIgnoreCase("LIST")) - { - ListCommand cmd = new ListCommand(this); - cmd.process(command); - } - - else if (commandName.equalsIgnoreCase("GROUP")) - { - GroupCommand cmd = new GroupCommand(this); - cmd.process(command); - } - - else if(commandName.equalsIgnoreCase("POST")) - { - PostCommand cmd = new PostCommand(this); - cmd.process(command); - } - - else if (commandName.equalsIgnoreCase("CHECK") - || commandName.equalsIgnoreCase("TAKETHIS")) - { - // untested, RFC2980 compliant - printStatus(400, "not accepting articles"); - return; - } - - else if (commandName.equalsIgnoreCase("IHAVE") - || commandName.equalsIgnoreCase("XREPLIC")) - { - // untested, RFC977 compliant - printStatus(435, "article not wanted - do not send it"); - return; - } - - else if (commandName.equalsIgnoreCase("XCREATEGROUP")) - { - return; - } - - else if (commandName.equalsIgnoreCase("SLAVE")) - { - // untested, RFC977 compliant - printStatus(202, "slave status noted"); - return; - } - - else if (commandName.equalsIgnoreCase("XINDEX")) - { - // untested, RFC2980 compliant - printStatus(418, "no tin-style index is available for this news group"); - return; - } - - else if (commandName.equalsIgnoreCase("DATE")) - { - printStatus(111, new SimpleDateFormat("yyyyMMddHHmmss") - .format(new Date())); - return; - } - - else if (commandName.equalsIgnoreCase("MODE")) - { - if (command[1].equalsIgnoreCase("READER")) - { - // untested, RFC2980 compliant - printStatus(200, "Hello, you can post"); - } - else if (command[1].equalsIgnoreCase("STREAM")) - { - printStatus(203, "Streaming is OK"); - } - else - printStatus(501, "Command not supported"); - } - - else if (commandName.equalsIgnoreCase("QUIT")) - { - // untested, RFC977 compliant - printStatus(205, "closing connection - goodbye!"); - exit(); - return; - } - - else if (commandName.equalsIgnoreCase("XSHUTDOWN")) - { - printStatus(205, "closing connection - goodbye!"); - exit(); - return; - } - - // X COMMANDS - else if(commandName.equalsIgnoreCase("XOVER") - || commandName.equalsIgnoreCase("OVER")) - { - OverCommand cmd = new OverCommand(this); - cmd.process(command); - } - - else - printStatus(501, "Command not supported"); - } - - /** - * Runloop of this Thread. - * @throws RuntimeException if this method is called directly. - */ - @Override - public void run() - { - assert !this.equals(Thread.currentThread()); - - try - { - printStatus(200, Config.getInstance().get("n3tpd.hostname", "localhost") - + " " + Main.VERSION + " news server ready - (posting ok)."); - } - catch (IOException e1) - { - exit(); - } - - while (!exit) - { - try - { - processCommand(readCommand()); - } - catch (SocketException e) - { - if (exit) - return; - exit(); - e.printStackTrace(); - } - catch (IOException e) - { - if (exit) - return; - exit(); - e.printStackTrace(); - } - catch (Throwable e) - { - if (exit) - return; - e.printStackTrace(); - // silently ignore - } - } - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/NNTPDaemon.java --- a/trunk/com/so/news/NNTPDaemon.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,64 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news; - -import java.io.IOException; -import java.net.ServerSocket; - -/** - * Server component of the n3tpd. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class NNTPDaemon extends Thread -{ - private ServerSocket socket; - - public NNTPDaemon(boolean aux) throws IOException - { - int port; - if(!aux) - port = Config.getInstance().get("n3tpd.port", 119); - else - port = Config.getInstance().get("n3tpd.auxport", 8080); - - int backlog = Config.getInstance().get("n3tpd.server.backlog", 10); - - // Create and bind the socket - socket = new ServerSocket(port, backlog); - } - - @Override - public void run() - { - System.out.println("Daemon listening on port " + socket.getLocalPort() + " ..."); - - while(isAlive() && !isInterrupted()) - { - try - { - new NNTPConnection(socket.accept()).start(); - } - catch (Exception e) - { - e.printStackTrace(); - } - } - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/ArticleCommand.java --- a/trunk/com/so/news/command/ArticleCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,124 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.io.IOException; -import java.text.SimpleDateFormat; -import java.util.Locale; -import java.util.Map; -import com.so.news.Debug; -import com.so.news.NNTPConnection; -import com.so.news.storage.Article; - -/** - * Class handling the ARTICLE command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class ArticleCommand extends Command -{ - public ArticleCommand(NNTPConnection connection) - { - super(connection); - } - - public boolean process(String[] command) throws IOException - { - String commandName = command[0]; - - // untested, RFC977 compliant - Article article = null; - if (command.length <= 1) - { - article = getCurrentArticle(); - if (article == null) - { - printStatus(420, "no current article has been selected"); - return true; - } - } - else if (command[1].matches(NNTPConnection.MESSAGE_ID_PATTERN)) - { - // Message-ID - article = Article.getByMessageID(command[1]); - if (article == null) - { - printStatus(430, "no such article found"); - return true; - } - } - else - { - // Message Number - try - { - int num = Integer.parseInt(command[1]); - article = Article.getByNumberInGroup(connection.getCurrentGroup(), num); - } - catch (Exception ex) - { - ex.printStackTrace(Debug.getInstance().getStream()); - System.err.println(ex.getLocalizedMessage()); - } - if (article == null) - { - printStatus(423, "no such article number in this group"); - return true; - } - setCurrentArticle(article); - } - - if (commandName.equalsIgnoreCase("ARTICLE")) - { - printStatus(220, article.getNumberInGroup() + " " + article.getMessageID() - + " article retrieved - head and body follow"); - Map header = article.getHeader(); - for(Map.Entry entry : header.entrySet()) - { - if(entry.getKey().equals("Date")) - { - SimpleDateFormat sdf = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.US); - printTextPart("Date: " + sdf.format(article.getDate())); - } - else - printTextPart(entry.getKey() + ": " + entry.getValue()); - } - println(""); - printText(article.getBody()); - } - else if (commandName.equalsIgnoreCase("HEAD")) - { - printStatus(500, "No longer supported! Use XOVER instead."); - return false; - } - else if (commandName.equalsIgnoreCase("BODY")) - { - printStatus(222, article.getNumberInGroup() + " " + article.getMessageID() - + " body"); - printText(article.getBody()); - } - else if (commandName.equalsIgnoreCase("STAT")) - { - printStatus(223, article.getNumberInGroup() + " " + article.getMessageID() - + " article retrieved - request text separately"); - } - return true; - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/CapabilitiesCommand.java --- a/trunk/com/so/news/command/CapabilitiesCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,50 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import com.so.news.NNTPConnection; - -/** - * The CAPABILITIES command allows a client to determine the - capabilities of the server at any given time. - - This command MAY be issued at any time; the server MUST NOT require - it to be issued in order to make use of any capability. The response - generated by this command MAY change during a session because of - other state information (which, in turn, may be changed by the - effects of other commands or by external events). An NNTP client is - only able to get the current and correct information concerning - available capabilities at any point during a session by issuing a - CAPABILITIES command at that point of that session and processing the - response. - * @author chris - */ -public class CapabilitiesCommand extends Command -{ - public CapabilitiesCommand(NNTPConnection connection) - { - super(connection); - } - - public boolean process(String[] command) - throws Exception - { - return false; - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/Command.java --- a/trunk/com/so/news/command/Command.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,106 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.io.IOException; -import java.util.List; -import com.so.news.NNTPConnection; -import com.so.news.storage.Article; -import com.so.news.storage.Group; - -/** - * Base class for all command handling classes. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public abstract class Command -{ - protected NNTPConnection connection; - - public Command(NNTPConnection connection) - { - this.connection = connection; - } - - protected static String NEWLINE = NNTPConnection.NEWLINE; - - protected List readText() - throws IOException - { - return connection.readText(); - } - - protected String readTextLine() throws IOException - { - return connection.readTextLine(); - } - - protected void printStatus(int status, String text) throws IOException - { - connection.printStatus(status, text); - } - - protected void printTextLine(CharSequence text) throws IOException - { - connection.printTextLine(text); - } - - protected void printTextPart(CharSequence text) throws IOException - { - connection.printTextPart(text); - } - - protected void printText(CharSequence text) throws IOException - { - connection.printText(text); - } - - protected void println(CharSequence text) throws IOException - { - connection.println(text); - } - - protected void flush() throws IOException - { - connection.flush(); - } - - protected Article getCurrentArticle() - { - return connection.getCurrentArticle(); - } - - protected Group getCurrentGroup() - { - return connection.getCurrentGroup(); - } - - protected void setCurrentArticle(Article current) - { - connection.setCurrentArticle(current); - } - - protected void setCurrentGroup(Group current) - { - connection.setCurrentGroup(current); - } - - public abstract boolean process(String[] command) - throws Exception; -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/GroupCommand.java --- a/trunk/com/so/news/command/GroupCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,60 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import com.so.news.NNTPConnection; -import com.so.news.storage.Group; - -/** - * Class handling the GROUP command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class GroupCommand extends Command -{ - public GroupCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) - throws Exception - { - // untested, RFC977 compliant - Group g = null; - if (command.length >= 2) - { - g = Group.getByName(command[1]); - } - if (g == null) - { - printStatus(411, "no such news group"); - return true; - } - else - { - setCurrentGroup(g); - - printStatus(211, g.getEstimatedArticleCount() + " " + g.getFirstArticle() - + " " + g.getLastArticle() + " " + g.getName() + " group selected"); - return true; - } - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/ListCommand.java --- a/trunk/com/so/news/command/ListCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,92 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.io.IOException; -import java.util.ArrayList; -import com.so.news.NNTPConnection; -import com.so.news.storage.Group; - -/** - * Class handling the LIST command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class ListCommand extends Command -{ - public ListCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) - throws Exception - { - if (command.length >= 2) - { - if (command[1].equalsIgnoreCase("OVERVIEW.FMT")) - { - printStatus(215, "information follows"); - printText("Subject:\nFrom:\nDate:\nMessage-ID:\nReferences:\nBytes:\nLines:"); - return true; - } - if (command[1].equalsIgnoreCase("NEWSGROUPS")) - { - printStatus(215, "information follows"); - ArrayList list = Group.getAll(); - for (Group g : list) - { - printTextLine(g.getName() + "\t" + "-"); - } - println("."); - flush(); - return true; - } - if (command[1].equalsIgnoreCase("SUBSCRIPTIONS")) - { - printStatus(215, "information follows"); - println("."); - flush(); - return true; - } - if (command[1].equalsIgnoreCase("EXTENSIONS")) - { - printStatus(202, "Supported NNTP extensions."); - printTextLine("LISTGROUP"); - println("."); - flush(); - return true; - } - return false; - } - printStatus(215, "list of newsgroups follows"); - for (Group g : Group.getAll()) - { - //if(g.getEstimatedArticleCount() <= 0) - // continue; - - printTextLine(g.getName() + " " + g.getLastArticle() + " " - + g.getFirstArticle() + " y"); - } - println("."); - flush(); - return true; - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/ListGroupCommand.java --- a/trunk/com/so/news/command/ListGroupCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,71 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.util.List; -import com.so.news.NNTPConnection; -import com.so.news.storage.Article; -import com.so.news.storage.Group; - -/** - * Class handling the LISTGROUP command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class ListGroupCommand extends Command -{ - public ListGroupCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) - throws Exception - { - String commandName = command[0]; - if (!commandName.equalsIgnoreCase("LISTGROUP")) - return false; - // untested, RFC977 complient - Group group = null; - if (command.length >= 2) - { - group = Group.getByName(command[1]); - } - else - { - group = getCurrentGroup(); - } - if (group == null) - { - printStatus(412, "Not currently in newsgroup"); - return true; - } - List
list = group.getAllArticles(); - printStatus(211, "list of article numbers follow"); // argh, bad english in - // RFC - for (Article a : list) - { - printTextLine("" + a.getNumberInGroup()); - } - println("."); - flush(); - return true; - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/NewGroupsCommand.java --- a/trunk/com/so/news/command/NewGroupsCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,68 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.io.IOException; -import java.util.ArrayList; -import com.so.news.NNTPConnection; -import com.so.news.storage.Group; - -/** - * Class handling the NEWGROUPS command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class NewGroupsCommand extends Command -{ - public NewGroupsCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) throws IOException - { - String commandName = command[0]; - if (!commandName.equalsIgnoreCase("NEWGROUPS")) - return false; - if (command.length != 3) - return false; - // untested, not RFC977 complient - try - { - // Timestamp date = new Timestamp ( new SimpleDateFormat ("yyMMdd - // HHmmss").parse(command[1] + " " + command[2] ).getTime()) ; - printStatus(231, "list of new newsgroups follows"); - ArrayList list = Group.getAll();// (date) ; - for (Group g : list) - { - printTextLine(g.getName() + " " + g.getLastArticle() + " " - + g.getFirstArticle() + " y"); - } - println("."); - flush(); - return true; - } - catch (Exception e) - { - printStatus(511, "listing failed - invalid date format"); - return true; - } - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/NextCommand.java --- a/trunk/com/so/news/command/NextCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,82 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.io.IOException; -import com.so.news.NNTPConnection; -import com.so.news.storage.Article; -import com.so.news.storage.Group; - -/** - * Class handling the NEXT and LAST command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class NextCommand extends Command -{ - public NextCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) throws IOException - { - String commandName = command[0]; - if (!(commandName.equalsIgnoreCase("NEXT") || commandName - .equalsIgnoreCase("LAST"))) - return false; - // untested, RFC977 complient - Article currA = getCurrentArticle(); - Group currG = getCurrentGroup(); - if (currA == null) - { - printStatus(420, "no current article has been selected"); - return true; - } - if (currG == null) - { - printStatus(412, "no newsgroup selected"); - return true; - } - Article article; - if (commandName.equalsIgnoreCase("NEXT")) - { - article = currA.nextArticleInGroup(); - if (article == null) - { - printStatus(421, "no next article in this group"); - return true; - } - } - else - { - article = currA.prevArticleInGroup(); - if (article == null) - { - printStatus(422, "no previous article in this group"); - return true; - } - } - setCurrentArticle(article); - printStatus(223, article.getNumberInGroup() + " " + article.getMessageID() - + " article retrieved - request text separately"); - return true; - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/OverCommand.java --- a/trunk/com/so/news/command/OverCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,183 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.text.SimpleDateFormat; -import java.util.Locale; - -import com.so.news.Debug; -import com.so.news.NNTPConnection; -import com.so.news.storage.Article; - -/** - * Class handling the OVER/XOVER command. - * - * Description of the XOVER command: - * - * XOVER [range] - * - * The XOVER command returns information from the overview - * database for the article(s) specified. - * - * The optional range argument may be any of the following: - * an article number - * an article number followed by a dash to indicate - * all following - * an article number followed by a dash followed by - * another article number - * - * If no argument is specified, then information from the - * current article is displayed. Successful responses start - * with a 224 response followed by the overview information - * for all matched messages. Once the output is complete, a - * period is sent on a line by itself. If no argument is - * specified, the information for the current article is - * returned. A news group must have been selected earlier, - * else a 412 error response is returned. If no articles are - * in the range specified, a 420 error response is returned - * by the server. A 502 response will be returned if the - * client only has permission to transfer articles. - * - * Each line of output will be formatted with the article number, - * followed by each of the headers in the overview database or the - * article itself (when the data is not available in the overview - * database) for that article separated by a tab character. The - * sequence of fields must be in this order: subject, author, - * date, message-id, references, byte count, and line count. Other - * optional fields may follow line count. Other optional fields may - * follow line count. These fields are specified by examining the - * response to the LIST OVERVIEW.FMT command. Where no data exists, - * a null field must be provided (i.e. the output will have two tab - * characters adjacent to each other). Servers should not output - * fields for articles that have been removed since the XOVER database - * was created. - * - * The LIST OVERVIEW.FMT command should be implemented if XOVER - * is implemented. A client can use LIST OVERVIEW.FMT to determine - * what optional fields and in which order all fields will be - * supplied by the XOVER command. - * - * Note that any tab and end-of-line characters in any header - * data that is returned will be converted to a space character. - * - * Responses: - * - * 224 Overview information follows - * 412 No news group current selected - * 420 No article(s) selected - * 502 no permission - * - * @author Christian Lins - */ -public class OverCommand extends Command -{ - public OverCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) - throws Exception - { - if(getCurrentGroup() == null) - { - printStatus(412, "No news group current selected"); - return false; - } - - // If no parameter was specified, show information about - // the currently selected article(s) - if(command.length == 1) - { - Article art = getCurrentArticle(); - if(art == null) - { - printStatus(420, "No article(s) selected"); - return false; - } - - String o = buildOverview(art, -1); - printText(o); - } - // otherwise print information about the specified range - else - { - int artStart = -1; - int artEnd = -1; - String[] nums = command[1].split("-"); - if(nums.length > 1) - { - try - { - artStart = Integer.parseInt(nums[0]); - } - catch(Exception e) - { - artStart = Integer.parseInt(command[1]); - } - try - { - artEnd = Integer.parseInt(nums[1]); - } - catch(Exception e) {} - } - - printStatus(224, "Overview information follows"); - for(int n = artStart; n <= artEnd; n++) - { - Article art = Article.getByNumberInGroup(getCurrentGroup(), n); - if(art == null) - { - Debug.getInstance().log("Article (gid=" + getCurrentGroup() + ", art=" + n + " is null!"); - } - else - { - printTextPart(buildOverview(art, n) + NEWLINE); - } - } - println("."); - flush(); - } - - return true; - } - - private String buildOverview(Article art, int nr) - { - SimpleDateFormat sdf = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.US); - StringBuilder overview = new StringBuilder(); - overview.append(nr); - overview.append('\t'); - overview.append(art.getHeader().get("Subject")); - overview.append('\t'); - overview.append(art.getHeader().get("From")); - overview.append('\t'); - overview.append(sdf.format(art.getDate())); - overview.append('\t'); - overview.append(art.getHeader().get("Message-ID")); - overview.append('\t'); - overview.append(art.getHeader().get("References")); - overview.append('\t'); - overview.append(art.getHeader().get("Bytes")); - overview.append('\t'); - overview.append(art.getHeader().get("Lines")); - - return overview.toString(); - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/command/PostCommand.java --- a/trunk/com/so/news/command/PostCommand.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,166 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.command; - -import java.io.IOException; -import java.sql.SQLException; -import java.util.Date; -import java.text.SimpleDateFormat; -import java.util.HashMap; -import java.util.Locale; - -import com.so.news.Config; -import com.so.news.Debug; -import com.so.news.NNTPConnection; -import com.so.news.storage.Article; -import com.so.news.storage.Database; - -/** - * Contains the code for the POST command. - * @author Christian Lins - * @author Dennis Schwerdel - */ -public class PostCommand extends Command -{ - public PostCommand(NNTPConnection conn) - { - super(conn); - } - - public boolean process(String[] command) throws IOException - { - printStatus(340, "send article to be posted. End with ."); - - // some initialization - Article article = new Article(); - int lineCount = 0; - long bodySize = 0; - long maxBodySize = Config.getInstance().get("n3tpd.article.maxsize", 1024) * 1024; // Size in bytes - - // begin with a stringbuilder body - StringBuilder body = new StringBuilder(); - HashMap header = new HashMap(); - - boolean isHeader = true; // are we in the header part - - String line = readTextLine(); - while(line != null) - { - bodySize += line.length(); - if(bodySize > maxBodySize) - { - printStatus(500, "article is too long"); - return false; - } - - if(!isHeader) - { // body - if(line.trim().equals(".")) - break; - - bodySize += line.length() + 1; - lineCount++; - body.append(line + NEWLINE); - } - - if(line.equals("")) - { - isHeader = false; // we finally met the blank line - // separating headers from body - } - - if(isHeader) - { // header - // split name and value and add the header to the map - int colon = line.indexOf(':'); - String fieldName = line.substring(0, colon).trim(); - String fieldValue = line.substring(colon + 1).trim(); - header.put(fieldName, fieldValue); - } - line = readTextLine(); // read a new line - } // end of input reading - - article.setBody(body.toString()); // set the article body - article.setHeader(header); // add the header entries for the article - - // Read the date header and fall back to the current date if it is not set - try - { - SimpleDateFormat sdf = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.US); - String date = header.get("DATE"); - if(date == null) - article.setDate(new Date()); - else - article.setDate(new Date(sdf.parse(date).getTime())) ; - } - catch (Exception e) - { - e.printStackTrace(Debug.getInstance().getStream()); - printStatus(541, "posting failed - invalid date format"); - return true; - } - - // check for a cancel command - if ( header.containsKey("Control") ) - { - String[] control = header.get("Control").split(" ") ; - if ( control.length >= 2 && control[0].equalsIgnoreCase("cancel") ) - { - // this article is a cancel-article, try to delete the old article - try - { - Article.getByMessageID(control[1]).delete(); - printStatus(240, "article posted ok - original article canceled"); // quite - return true; // quit, do not actually post this article since it - } - catch (Exception e) - { - e.printStackTrace(); - printStatus(441, "posting failed - original posting not found"); - return true; - } - } - } - - // set some headers - header.put("Message-ID", article.getMessageID()); - header.put("Lines", "" + lineCount); - header.put("Bytes", "" + bodySize); - - // if needed, set an empty references header, that means this is - // a initial posting - if (!header.containsKey("References")) - header.put("References", ""); - - // try to create the article in the database - try - { - Database.getInstance().addArticle(article); - printStatus(240, "article posted ok"); - } - catch(SQLException ex) - { - System.err.println(ex.getLocalizedMessage()); - ex.printStackTrace(Debug.getInstance().getStream()); - printStatus(500, "internal server error"); - } - - return true; - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/io/Resource.java --- a/trunk/com/so/news/io/Resource.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,123 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.io; - -import java.io.BufferedReader; -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.net.URL; -import java.nio.charset.Charset; - -/** - * Provides method for loading of resources. - * @author Christian Lins - */ -public class Resource -{ - /** - * Loads a file as array of byte. As the file is completely loaded into - * memory this method should only be used with small files. - * @param file - * @return - */ - public static byte[] getBytes(File file) - { - try - { - FileInputStream in = new FileInputStream(file); - byte[] buffer = new byte[(int)file.length()]; - - in.read(buffer); - - return buffer; - } - catch(IOException ex) - { - System.err.println(ex.getLocalizedMessage()); - return null; - } - } - - /** - * Loads a resource and returns it as URL reference. - * The Resource's classloader is used to load the resource, not - * the System's ClassLoader so it may be safe to use this method - * in a sandboxed environment. - * @return - */ - public static URL getAsURL(String name) - { - return Resource.class.getClassLoader().getResource(name); - } - - /** - * Loads a resource and returns an InputStream to it. - * @param name - * @return - */ - public static InputStream getAsStream(String name) - { - try - { - URL url = getAsURL(name); - return url.openStream(); - } - catch(IOException e) - { - e.printStackTrace(); - return null; - } - } - - /** - * Loads a plain text resource. - * @param withNewline If false all newlines are removed from the - * return String - */ - public static String getAsString(String name, boolean withNewline) - { - try - { - BufferedReader in = new BufferedReader( - new InputStreamReader(getAsStream(name), Charset.forName("UTF-8"))); - StringBuffer buf = new StringBuffer(); - - for(;;) - { - String line = in.readLine(); - if(line == null) - break; - - buf.append(line); - if(withNewline) - buf.append('\n'); - } - - return buf.toString(); - } - catch(Exception e) - { - e.printStackTrace(); - return null; - } - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/storage/Article.java --- a/trunk/com/so/news/storage/Article.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,307 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.storage; - -import java.sql.ResultSet; -import java.sql.SQLException; -import java.util.Date; -import java.util.HashMap; -import java.util.Map; -import java.util.Map.Entry; -import java.util.UUID; - -import com.so.news.Config; -import com.so.news.Debug; - -/** - * Represents a newsgroup article. - * @author Christian Lins - * @author Denis Schwerdel - */ -public class Article -{ - /** - * Loads the Article identified by the given ID from the Database. - * @param messageID - * @return null if Article is not found or if an error occurred. - */ - public static Article getByMessageID(String messageID) - { - try - { - return Database.getInstance().getArticle(messageID); - } - catch(SQLException ex) - { - ex.printStackTrace(Debug.getInstance().getStream()); - return null; - } - } - - public static Article getByNumberInGroup(Group group, int number) - throws SQLException - { - long gid = group.getID(); - return Database.getInstance().getArticle(gid, number); // Is number her correct? - } - - private String body = ""; - private long groupID = -1; - private Map header = new HashMap(); - private int numberInGroup = -1; - private String msgID = null; - - /** - * Default constructor. - */ - public Article() - { - } - - /** - * Creates a new Article object using the date from the given - * ResultSet. It is expected that ResultSet.next() was already - * called by the Database class. - * This construction has only package visibility. - * @param rs - */ - Article(ResultSet rs) - throws SQLException - { - this.body = rs.getString("body"); - this.msgID = rs.getString("message_id"); - - // Parse the header - parseHeader(rs.getString("header")); - } - - /** - * Parses the header fields and puts them into a map for faster access. - * TODO: There could be fields that go over more than one line, some - * bad clients do create them. - * @param hsrc - */ - private void parseHeader(String hsrc) - { - String[] lines = hsrc.split("\n"); - - for(String line : lines) - { - String[] kv = line.split(":"); - if(kv.length < 2) - { - Debug.getInstance().log("Invalid header field: " + line); - continue; - } - else - { - // Set value in the header hash map - String value = kv[1]; - for(int n = 2; n < kv.length; n++) - value += ":" + kv[n]; - this.header.put(kv[0], value); - } - } - } - - /** - * Returnes the next Article in the group of this Article. - * @return - */ - public Article nextArticleInGroup() - { - return null; - } - - /** - * Returns the previous Article in the group of this Article. - * @return - */ - public Article prevArticleInGroup() - { - return null; - } - - /** - * Generates a message id for this article and sets it into - * the header HashMap. - */ - private String generateMessageID() - { - this.msgID = "<" + UUID.randomUUID() + "@" - + Config.getInstance().get("n3tpd.hostname", "localhost") + ">"; - - this.header.put("Message-ID", msgID); - - return msgID; - } - - /** - * Tries to delete this article. - * @return false if the article could not be deleted, otherwise true - */ - public boolean delete() - { - return false; - } - - /** - * Checks if all necessary header fields are within this header. - */ - private void validateHeader() - { - // Forces a MessageID creation if not existing - getMessageID(); - - // Check if the references are correct... - String rep = header.get("In-Reply-To"); - if(rep == null) // Some clients use only references instead of In-Reply-To - return; //rep = header.get("References"); - - String ref = getMessageID(); - - if(rep != null && !rep.equals("")) - { - Article art = null; //TODO // getByMessageID(rep, articleDir); - if(art != null) - { - ref = art.header.get("References") + " " + rep; - } - } - header.put("References", ref); - } - - /** - * Returns the body string. - */ - public String getBody() - { - return body; - } - - /** - * @return Numerical ID of the associated Group. - */ - long getGroupID() - { - if(groupID == -1) // If the GroupID was not determined yet - { - // Determining GroupID - String newsgroups = this.header.get("Newsgroups"); - if(newsgroups != null) - { - String[] newsgroup = newsgroups.split(","); - // Crossposting is not supported - try - { - Group group; - if(newsgroup.length > 0) - group = Database.getInstance().getGroup(newsgroup[0].trim()); - else - group = Database.getInstance().getGroup(newsgroups.trim()); - // TODO: What to do if Group does not exist? - this.groupID = group.getID(); - } - catch(SQLException ex) - { - ex.printStackTrace(Debug.getInstance().getStream()); - System.err.println(ex.getLocalizedMessage()); - } - } - else - System.err.println("Should never happen: Article::getGroupID"); - } - return this.groupID; - } - - public void setBody(String body) - { - this.body = body; - } - - public int getNumberInGroup() - { - return this.numberInGroup; - } - - public void setHeader(HashMap header) - { - this.header = header; - } - - public void setNumberInGroup(int id) - { - this.numberInGroup = id; - } - - public String getMessageID() - { - if(msgID == null) - msgID = generateMessageID(); - return msgID; - } - - /** - * @return Header source code of this Article. - */ - public String getHeaderSource() - { - StringBuffer buf = new StringBuffer(); - - for(Entry entry : this.header.entrySet()) - { - buf.append(entry.getKey()); - buf.append(":"); - buf.append(entry.getValue()); - buf.append("\n"); - } - - return buf.toString(); - } - - public Map getHeader() - { - return this.header; - } - - public Date getDate() - { - try - { - String date = this.header.get("Date"); - return new Date(Date.parse(date)); - } - catch(Exception e) - { - e.printStackTrace(Debug.getInstance().getStream()); - return null; - } - } - - public void setDate(Date date) - { - this.header.put("Date", date.toString()); - } - - @Override - public String toString() - { - return getMessageID(); - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/storage/Database.java --- a/trunk/com/so/news/storage/Database.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,327 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.storage; - -import java.sql.Connection; -import java.sql.DriverManager; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Statement; - -import java.util.zip.CRC32; -import com.so.news.Config; -import com.so.news.util.StringTemplate; - -/** - * Database abstraction class. - * @author Christian Lins (christian.lins@web.de) - */ -public class Database -{ - private static Database instance = null; - - /** - * Initializes the Database subsystem, e.g. loading a JDBC driver and - * connection to the Database Managment System. - * This method is called when the daemon starts up or at the first - * call to Database.getInstance(). - * @throws java.lang.Exception - */ - public static void arise() - throws Exception - { - // Tries to load the Database driver and establish a connection. - if(instance == null) - instance = new Database(); - } - - /** - * @return Instance of the current Database backend. Returns null if an error - * has occurred. - */ - public static Database getInstance() - { - try - { - arise(); - return instance; - } - catch(Exception ex) - { - ex.printStackTrace(); - return null; - } - } - - private Connection conn = null; - - /** - * Private constructor. - * @throws java.lang.Exception - */ - private Database() - throws Exception - { - Class.forName( - Config.getInstance().get("n3tpd.storage.dbmsdriver", "")); - this.conn = DriverManager.getConnection( - Config.getInstance().get("n3tpd.storage.database", ""), - Config.getInstance().get("n3tpd.storage.user", "n3tpd_user"), - Config.getInstance().get("n3tpd.storage.password", "")); - this.conn.setAutoCommit(false); - } - - /** - * Adds an article to the database. - * @param article - * @return - * @throws java.sql.SQLException - */ - public boolean addArticle(Article article) - throws SQLException - { - Statement stmt = this.conn.createStatement(); - - String sql0 = "START TRANSACTION"; - String sql1 = "INSERT INTO articles (message_id,header,body)" + - "VALUES('%mid', '%header', '%body')"; - StringTemplate tmpl = new StringTemplate(sql1); - tmpl.set("body", article.getBody()); - tmpl.set("mid", article.getMessageID()); - tmpl.set("header", article.getHeaderSource()); - sql1 = tmpl.toString(); - - String sql2 = "COMMIT"; - - // Add statements as batch - stmt.addBatch(sql0); - stmt.addBatch(sql1); - - // TODO: For each newsgroup add a reference - String sql = "INSERT INTO postings (group_id, article_id, article_index)" + - "VALUES (%gid, (SELECT article_id FROM articles WHERE message_id = '%mid')," + - " %idx)"; - - tmpl = new StringTemplate(sql); - tmpl.set("gid", article.getGroupID()); - tmpl.set("mid", article.getMessageID()); - tmpl.set("idx", getMaxArticleIndex() + 1); - stmt.addBatch(tmpl.toString()); - - // Commit - stmt.addBatch(sql2); - - // And execute the batch - stmt.executeBatch(); - - return true; - } - - /** - * Adds a group to the Database. - * @param name - * @throws java.sql.SQLException - */ - public boolean addGroup(String name) - throws SQLException - { - CRC32 crc = new CRC32(); - crc.update(name.getBytes()); - - long id = crc.getValue(); - - Statement stmt = conn.createStatement(); - return 1 == stmt.executeUpdate("INSERT INTO Groups (ID, Name) VALUES (" + id + ", '" + name + "')"); - } - - public void delete(Article article) - { - - } - - public void delete(Group group) - { - - } - - public Article getArticle(String messageID) - throws SQLException - { - Statement stmt = this.conn.createStatement(); - ResultSet rs = - stmt.executeQuery("SELECT * FROM articles WHERE message_id = '" + messageID + "'"); - - return new Article(rs); - } - - public Article getArticle(long gid, long article_id) - throws SQLException - { - Statement stmt = this.conn.createStatement(); - String sql = "SELECT * FROM articles WHERE article_id = " + - "(SELECT article_id FROM postings WHERE " + - "group_id = " + gid + " AND article_id = " + article_id +")"; - ResultSet rs = - stmt.executeQuery(sql); - - if(rs.next()) - return new Article(rs); - else - return null; - } - - public ResultSet getArticles() - throws SQLException - { - Statement stmt = conn.createStatement(); - return stmt.executeQuery("SELECT * FROM articles"); - } - - /** - * Reads all Groups from the Database. - * @return - * @throws java.sql.SQLException - */ - public ResultSet getGroups() - throws SQLException - { - Statement stmt = conn.createStatement(); - ResultSet rs = stmt.executeQuery("SELECT * FROM groups"); - - return rs; - } - - /** - * Returns the Group that is identified by the name. - * @param name - * @return - * @throws java.sql.SQLException - */ - public Group getGroup(String name) - throws SQLException - { - Statement stmt = this.conn.createStatement(); - String sql = "SELECT group_id FROM groups WHERE Name = '%name'"; - StringTemplate tmpl = new StringTemplate(sql); - tmpl.set("name", name); - - ResultSet rs = stmt.executeQuery(tmpl.toString()); - - if(!rs.next()) - return null; - else - { - long id = rs.getLong("group_id"); - return new Group(name, id); - } - } - - public int getMaxArticleIndex() - throws SQLException - { - Statement stmt = conn.createStatement(); - ResultSet rs = stmt.executeQuery( - "SELECT Max(article_index) FROM postings"); - - if(!rs.next()) - return 0; - else - return rs.getInt(1); - } - - public int getLastArticleNumber(Group group) - throws SQLException - { - Statement stmt = conn.createStatement(); - ResultSet rs = stmt.executeQuery( - "SELECT Max(article_index) FROM postings WHERE group_id = " + group.getID()); - - if(!rs.next()) - return 0; - else - return rs.getInt(1); - } - - public int getFirstArticleNumber(Group group) - throws SQLException - { - Statement stmt = conn.createStatement(); - ResultSet rs = stmt.executeQuery( - "SELECT Min(article_index) FROM postings WHERE group_id = " + group.getID()); - - if(!rs.next()) - return 0; - else - return rs.getInt(1); - } - - /** - * Returns a group name identified by the given id. - * @param id - * @return - * @throws java.sql.SQLException - */ - public String getGroup(int id) - throws SQLException - { - Statement stmt = conn.createStatement(); - ResultSet rs = stmt.executeQuery( - "SELECT name FROM groups WHERE group_id = '" + id + "'"); - - if(rs.next()) - { - return rs.getString(1); - } - else - return null; - } - - public Article getOldestArticle() - throws SQLException - { - Statement stmt = conn.createStatement(); - ResultSet rs = - stmt.executeQuery("SELECT * FROM Articles WHERE Date = (SELECT Min(Date) FROM Articles)"); - - if(rs.next()) - return new Article(rs); - else - return null; - } - - /** - * Checks if there is a group with the given name in the Database. - * @param name - * @return - * @throws java.sql.SQLException - */ - public boolean isGroupExisting(String name) - throws SQLException - { - Statement stmt = this.conn.createStatement(); - ResultSet rs = stmt.executeQuery("SELECT * FROM Groups WHERE Name = '" + name + "'"); - - return rs.next(); - } - - public void updateArticle(Article article) - { - - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/storage/Group.java --- a/trunk/com/so/news/storage/Group.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,142 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.storage; - -import java.sql.ResultSet; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; -import com.so.news.Debug; - -/** - * Represents a logical Group within this newsserver. - * @author Christian Lins - */ -public class Group -{ - private long id; - private String name; - - /** - * Private constructor. - * @param name - * @param id - */ - Group(String name, long id) - { - this.id = id; - this.name = name; - } - - /** - * Returns a Group identified by its full name. - * @param name - * @return - */ - public static Group getByName(String name) - { - try - { - return Database.getInstance().getGroup(name); - } - catch(SQLException ex) - { - System.err.println(ex.getLocalizedMessage()); - ex.printStackTrace(Debug.getInstance().getStream()); - return null; - } - } - - /** - * Returns a list of all groups this server handles. - * @return - */ - public static ArrayList getAll() - { - ArrayList buffer = new ArrayList(); - - try - { - ResultSet rs = Database.getInstance().getGroups(); - - while(rs.next()) - { - String name = rs.getString("name"); - long id = rs.getLong("group_id"); - - Group group = new Group(name, id); - buffer.add(group); - } - } - catch(SQLException ex) - { - ex.printStackTrace(Debug.getInstance().getStream()); - System.err.println(ex.getLocalizedMessage()); - } - - return buffer; - } - - public List
getAllArticles() - throws SQLException - { - return getAllArticles(getFirstArticle(), getLastArticle()); - } - - public List
getAllArticles(int first, int last) - { - return null; - } - - public int getFirstArticle() - throws SQLException - { - return Database.getInstance().getFirstArticleNumber(this); - } - - public long getID() - { - return id; - } - - public int getLastArticle() - throws SQLException - { - return Database.getInstance().getLastArticleNumber(this); - } - - public String getName() - { - return name; - } - - public void setName(String name) - { - this.name = name; - } - - public int getEstimatedArticleCount() - throws SQLException - { - if (getLastArticle() < getFirstArticle()) - return 0; - return getLastArticle() - getFirstArticle() + 1; - } - -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/storage/Purger.java --- a/trunk/com/so/news/storage/Purger.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,96 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.storage; - -import java.util.Date; - -import com.so.news.Config; -import com.so.news.Debug; - -/** - * The purger is started in configurable intervals to search - * for old messages that can be purged. - * @author Christian Lins - */ -public class Purger extends Thread -{ - private int interval; - - public Purger() - { - setDaemon(true); // Daemons run only along with the main thread - setPriority(Thread.MIN_PRIORITY); - - this.interval = Config.getInstance().get("n3tpd.article.lifetime", 30) * 24 * 60 * 60 * 1000; // Milliseconds - if(this.interval < 0) - this.interval = Integer.MAX_VALUE; - } - - /** - * Runloop of this Purger class. - */ - @Override - public void run() - { - for(;;) - { - purge(); - - try - { - sleep(interval); - } - catch(InterruptedException e) - { - e.printStackTrace(Debug.getInstance().getStream()); - } - } - } - - /** - * Loops through all messages and deletes them if their time - * has come. - */ - private void purge() - { - Debug.getInstance().log("Purging old messages..."); - - try - { - for(;;) - { - Article art = null; //Database.getInstance().getOldestArticle(); - if(art == null) // No articles in the database - break; - - if(art.getDate().getTime() < (new Date().getTime() + this.interval)) - { - Database.getInstance().delete(art); - Debug.getInstance().log("Deleted: " + art); - } - else - break; - } - } - catch(Exception ex) - { - ex.printStackTrace(); - } - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/com/so/news/util/StringTemplate.java --- a/trunk/com/so/news/util/StringTemplate.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,77 +0,0 @@ -/* - * StarOffice News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package com.so.news.util; - -import java.util.HashMap; - -/** - * Class that allows simple String template handling. - * @author Christian Lins (christian.lins@web.de) - */ -public class StringTemplate -{ - private String str = null; - private String templateDelimiter = "%"; - private HashMap templateValues = new HashMap(); - - public StringTemplate(String str, String templateDelimiter) - { - this.str = str; - this.templateDelimiter = templateDelimiter; - } - - public StringTemplate(String str) - { - this(str, "%"); - } - - public void set(String template, String value) - { - this.templateValues.put(template, value); - } - - public void set(String template, long value) - { - set(template, Long.toString(value)); - } - - public void set(String template, double value) - { - set(template, Double.toString(value)); - } - - public void set(String template, Object obj) - { - set(template, obj.toString()); - } - - @Override - public String toString() - { - String ret = new String(str); - - for(String key : this.templateValues.keySet()) - { - String value = this.templateValues.get(key); - ret = ret.replace(templateDelimiter + key, value); - } - - return ret; - } -} diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/helpers/tbl_mysql6_tmpl.sql --- a/trunk/helpers/tbl_mysql6_tmpl.sql Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,39 +0,0 @@ -CREATE DATABASE staroffice_news; - -CREATE TABLE groups -( - group_id SERIAL, - name VARCHAR(80) NOT NULL, - flags INTEGER DEFAULT 0 NOT NULL -); - -CREATE UNIQUE INDEX name_id_index ON groups (name); - -CREATE TABLE articles -( - article_id SERIAL, - message_id TEXT, - header TEXT, - body TEXT -); - -CREATE UNIQUE INDEX article_message_index ON articles (message_id(255)); - -CREATE TABLE postings -( - group_id INTEGER, - article_id INTEGER, - article_index INTEGER NOT NULL -); - -CREATE UNIQUE INDEX posting_article_index ON postings (article_id); - -CREATE TABLE subscriptions -( - group_id INTEGER -); - -CREATE TABLE overview -( - header TEXT -); diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/rfc3977.txt --- a/trunk/rfc3977.txt Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6998 +0,0 @@ - -Network Working Group C. Feather -Request for Comments: 3977 THUS plc -Obsoletes: 977 October 2006 -Updates: 2980 -Category: Standards Track - - - Network News Transfer Protocol (NNTP) - -Status of This Memo - - This document specifies an Internet standards track protocol for the - Internet community, and requests discussion and suggestions for - improvements. Please refer to the current edition of the "Internet - Official Protocol Standards" (STD 1) for the standardization state - and status of this protocol. Distribution of this memo is unlimited. - -Copyright Notice - - Copyright (C) The Internet Society (2006). - -Abstract - - The Network News Transfer Protocol (NNTP) has been in use in the - Internet for a decade, and remains one of the most popular protocols - (by volume) in use today. This document is a replacement for - RFC 977, and officially updates the protocol specification. It - clarifies some vagueness in RFC 977, includes some new base - functionality, and provides a specific mechanism to add standardized - extensions to NNTP. - -Table of Contents - - 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 - 1.1. Author's Note . . . . . . . . . . . . . . . . . . . . . . 4 - 2. Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 3. Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . 6 - 3.1. Commands and Responses . . . . . . . . . . . . . . . . . 6 - 3.1.1. Multi-line Data Blocks . . . . . . . . . . . . . . . . 8 - 3.2. Response Codes . . . . . . . . . . . . . . . . . . . . . 9 - 3.2.1. Generic Response Codes . . . . . . . . . . . . . . . 10 - 3.2.1.1. Examples . . . . . . . . . . . . . . . . . . . . 12 - 3.3. Capabilities and Extensions . . . . . . . . . . . . . . . 14 - 3.3.1. Capability Descriptions . . . . . . . . . . . . . . . 14 - 3.3.2. Standard Capabilities . . . . . . . . . . . . . . . . 15 - 3.3.3. Extensions . . . . . . . . . . . . . . . . . . . . . 16 - 3.3.4. Initial IANA Register . . . . . . . . . . . . . . . . 18 - 3.4. Mandatory and Optional Commands . . . . . . . . . . . . . 20 - - - -Feather Standards Track [Page 1] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - 3.4.1. Reading and Transit Servers . . . . . . . . . . . . . 21 - 3.4.2. Mode Switching . . . . . . . . . . . . . . . . . . . 21 - 3.5. Pipelining . . . . . . . . . . . . . . . . . . . . . . . 22 - 3.5.1. Examples . . . . . . . . . . . . . . . . . . . . . . 23 - 3.6. Articles . . . . . . . . . . . . . . . . . . . . . . . . 24 - 4. The WILDMAT Format . . . . . . . . . . . . . . . . . . . . . 25 - 4.1. Wildmat Syntax . . . . . . . . . . . . . . . . . . . . . 26 - 4.2. Wildmat Semantics . . . . . . . . . . . . . . . . . . . . 26 - 4.3. Extensions . . . . . . . . . . . . . . . . . . . . . . . 27 - 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . 27 - 5. Session Administration Commands . . . . . . . . . . . . . . . 28 - 5.1. Initial Connection . . . . . . . . . . . . . . . . . . . 28 - 5.2. CAPABILITIES . . . . . . . . . . . . . . . . . . . . . . 29 - 5.3. MODE READER . . . . . . . . . . . . . . . . . . . . . . . 32 - 5.4. QUIT . . . . . . . . . . . . . . . . . . . . . . . . . . 34 - 6. Article Posting and Retrieval . . . . . . . . . . . . . . . . 35 - 6.1. Group and Article Selection . . . . . . . . . . . . . . . 36 - 6.1.1. GROUP . . . . . . . . . . . . . . . . . . . . . . . . 36 - 6.1.2. LISTGROUP . . . . . . . . . . . . . . . . . . . . . . 39 - 6.1.3. LAST . . . . . . . . . . . . . . . . . . . . . . . . 42 - 6.1.4. NEXT . . . . . . . . . . . . . . . . . . . . . . . . 44 - 6.2. Retrieval of Articles and Article Sections . . . . . . . 45 - 6.2.1. ARTICLE . . . . . . . . . . . . . . . . . . . . . . . 46 - 6.2.2. HEAD . . . . . . . . . . . . . . . . . . . . . . . . 49 - 6.2.3. BODY . . . . . . . . . . . . . . . . . . . . . . . . 51 - 6.2.4. STAT . . . . . . . . . . . . . . . . . . . . . . . . 53 - 6.3. Article Posting . . . . . . . . . . . . . . . . . . . . . 56 - 6.3.1. POST . . . . . . . . . . . . . . . . . . . . . . . . 56 - 6.3.2. IHAVE . . . . . . . . . . . . . . . . . . . . . . . . 58 - 7. Information Commands . . . . . . . . . . . . . . . . . . . . 61 - 7.1. DATE . . . . . . . . . . . . . . . . . . . . . . . . . . 61 - 7.2. HELP . . . . . . . . . . . . . . . . . . . . . . . . . . 62 - 7.3. NEWGROUPS . . . . . . . . . . . . . . . . . . . . . . . . 63 - 7.4. NEWNEWS . . . . . . . . . . . . . . . . . . . . . . . . . 64 - 7.5. Time . . . . . . . . . . . . . . . . . . . . . . . . . . 65 - 7.5.1. Examples . . . . . . . . . . . . . . . . . . . . . . 66 - 7.6. The LIST Commands . . . . . . . . . . . . . . . . . . . . 66 - 7.6.1. LIST . . . . . . . . . . . . . . . . . . . . . . . . 67 - 7.6.2. Standard LIST Keywords . . . . . . . . . . . . . . . 69 - 7.6.3. LIST ACTIVE . . . . . . . . . . . . . . . . . . . . . 70 - 7.6.4. LIST ACTIVE.TIMES . . . . . . . . . . . . . . . . . . 71 - 7.6.5. LIST DISTRIB.PATS . . . . . . . . . . . . . . . . . . 72 - 7.6.6. LIST NEWSGROUPS . . . . . . . . . . . . . . . . . . . 73 - 8. Article Field Access Commands . . . . . . . . . . . . . . . . 73 - 8.1. Article Metadata . . . . . . . . . . . . . . . . . . . . 74 - 8.1.1. The :bytes Metadata Item . . . . . . . . . . . . . . 74 - 8.1.2. The :lines Metadata Item . . . . . . . . . . . . . . 75 - 8.2. Database Consistency . . . . . . . . . . . . . . . . . . 75 - - - -Feather Standards Track [Page 2] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - 8.3. OVER . . . . . . . . . . . . . . . . . . . . . . . . . . 76 - 8.4. LIST OVERVIEW.FMT . . . . . . . . . . . . . . . . . . . . 81 - 8.5. HDR . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 - 8.6. LIST HEADERS . . . . . . . . . . . . . . . . . . . . . . 87 - 9. Augmented BNF Syntax for NNTP . . . . . . . . . . . . . . . . 90 - 9.1. Introduction . . . . . . . . . . . . . . . . . . . . . . 90 - 9.2. Commands . . . . . . . . . . . . . . . . . . . . . . . . 92 - 9.3. Command Continuation . . . . . . . . . . . . . . . . . . 93 - 9.4. Responses . . . . . . . . . . . . . . . . . . . . . . . . 93 - 9.4.1. Generic Responses . . . . . . . . . . . . . . . . . . 93 - 9.4.2. Initial Response Line Contents . . . . . . . . . . . 94 - 9.4.3. Multi-line Response Contents . . . . . . . . . . . . 94 - 9.5. Capability Lines . . . . . . . . . . . . . . . . . . . . 95 - 9.6. LIST Variants . . . . . . . . . . . . . . . . . . . . . . 96 - 9.7. Articles . . . . . . . . . . . . . . . . . . . . . . . . 97 - 9.8. General Non-terminals . . . . . . . . . . . . . . . . . . 97 - 9.9. Extensions and Validation . . . . . . . . . . . . . . . . 99 - 10. Internationalisation Considerations . . . . . . . . . . . . .100 - 10.1. Introduction and Historical Situation . . . . . . . . . .100 - 10.2. This Specification . . . . . . . . . . . . . . . . . . .101 - 10.3. Outstanding Issues . . . . . . . . . . . . . . . . . . .102 - 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .103 - 12. Security Considerations . . . . . . . . . . . . . . . . . . .103 - 12.1. Personal and Proprietary Information . . . . . . . . . .104 - 12.2. Abuse of Server Log Information . . . . . . . . . . . . .104 - 12.3. Weak Authentication and Access Control . . . . . . . . .104 - 12.4. DNS Spoofing . . . . . . . . . . . . . . . . . . . . . .104 - 12.5. UTF-8 Issues . . . . . . . . . . . . . . . . . . . . . .105 - 12.6. Caching of Capability Lists . . . . . . . . . . . . . . .106 - 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . .107 - 14. References . . . . . . . . . . . . . . . . . . . . . . . . .110 - 14.1. Normative References . . . . . . . . . . . . . . . . . .110 - 14.2. Informative References . . . . . . . . . . . . . . . . .110 - A. Interaction with Other Specifications . . . . . . . . . . . .112 - A.1. Header Folding . . . . . . . . . . . . . . . . . . . . .112 - A.2. Message-IDs . . . . . . . . . . . . . . . . . . . . . . .112 - A.3. Article Posting . . . . . . . . . . . . . . . . . . . . .114 - B. Summary of Commands . . . . . . . . . . . . . . . . . . . . .115 - C. Summary of Response Codes . . . . . . . . . . . . . . . . . .117 - D. Changes from RFC 977 . . . . . . . . . . . . . . . . . . . .121 - -1. Introduction - - This document specifies the Network News Transfer Protocol (NNTP), - which is used for the distribution, inquiry, retrieval, and posting - of Netnews articles using a reliable stream-based mechanism. For - news-reading clients, NNTP enables retrieval of news articles that - - - - -Feather Standards Track [Page 3] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - are stored in a central database, giving subscribers the ability to - select only those articles they wish to read. - - The Netnews model provides for indexing, cross-referencing, and - expiration of aged messages. NNTP is designed for efficient - transmission of Netnews articles over a reliable full duplex - communication channel. - - Although the protocol specification in this document is largely - compatible with the version specified in RFC 977 [RFC977], a number - of changes are summarised in Appendix D. In particular: - - o the default character set is changed from US-ASCII [ANSI1986] to - UTF-8 [RFC3629] (note that US-ASCII is a subset of UTF-8); - - o a number of commands that were optional in RFC 977 or that have - been taken from RFC 2980 [RFC2980] are now mandatory; and - - o a CAPABILITIES command has been added to allow clients to - determine what functionality is available from a server. - - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", - "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this - document are to be interpreted as described in RFC 2119 [RFC2119]. - - An implementation is not compliant if it fails to satisfy one or more - of the MUST requirements for this protocol. An implementation that - satisfies all the MUST and all the SHOULD requirements for its - protocols is said to be "unconditionally compliant"; one that - satisfies all the MUST requirements but not all the SHOULD - requirements for NNTP is said to be "conditionally compliant". - - For the remainder of this document, the terms "client" and "client - host" refer to a host making use of the NNTP service, while the terms - "server" and "server host" refer to a host that offers the NNTP - service. - -1.1. Author's Note - - This document is written in XML using an NNTP-specific DTD. Custom - software is used to convert this to RFC 2629 [RFC2629] format, and - then the public "xml2rfc" package to further reduce this to text, - nroff source, and HTML. - - No perl was used in producing this document. - - - - - - -Feather Standards Track [Page 4] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -2. Notation - - The following notational conventions are used in this document. - - UPPERCASE indicates literal text to be included in the - command. - - lowercase indicates a token described elsewhere. - - [brackets] indicate that the enclosed material is optional. - - elliptical indicates that the argument may be repeated any - ... marks number of times (it must occur at least once). - - vertical|bar indicates a choice of two mutually exclusive - arguments (exactly one must be provided). - - The name "message-id" for a command or response argument indicates - that it is the message-id of an article as described in Section 3.6, - including the angle brackets. - - The name "wildmat" for an argument indicates that it is a wildmat as - defined in Section 4. If the argument does not meet the requirements - of that section (for example, if it does not fit the grammar of - Section 4.1), the NNTP server MAY place some interpretation on it - (not specified by this document) or otherwise MUST treat it as a - syntax error. - - Responses for each command will be described in tables listing the - required format of a response followed by the meaning that should be - ascribed to that response. - - The terms "NUL", "TAB", "LF", "CR, and "space" refer to the octets - %x00, %x09, %x0A, %x0D, and %x20, respectively (that is, the octets - with those codes in US-ASCII [ANSI1986] and thus in UTF-8 [RFC3629]). - The term "CRLF" or "CRLF pair" means the sequence CR immediately - followed by LF (that is, %x0D.0A). A "printable US-ASCII character" - is an octet in the range %x21-7E. Quoted characters refer to the - octets with those codes in US-ASCII (so "." and "<" refer to %x2E and - %x3C) and will always be printable US-ASCII characters; similarly, - "digit" refers to the octets %x30-39. - - A "keyword" MUST consist only of US-ASCII letters, digits, and the - characters dot (".") and dash ("-") and MUST begin with a letter. - Keywords MUST be at least three characters in length. - - - - - - -Feather Standards Track [Page 5] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Examples in this document are not normative but serve to illustrate - usages, arguments, and responses. In the examples, a "[C]" will be - used to represent the client host and an "[S]" will be used to - represent the server host. Most of the examples do not rely on a - particular server state. In some cases, however, they do assume that - the currently selected newsgroup (see the GROUP command, - Section 6.1.1) is invalid; when so, this is indicated at the start of - the example. Examples may use commands or other keywords not defined - in this specification (such as an XENCRYPT command). These will be - used to illustrate some point and do not imply that any such command - is defined elsewhere or needs to exist in any particular - implementation. - - Terms that might be read as specifying details of a client or server - implementation, such as "database", are used simply to ease - description. Provided that implementations conform to the protocol - and format specifications in this document, no specific technique is - mandated. - -3. Basic Concepts - -3.1. Commands and Responses - - NNTP operates over any reliable bi-directional 8-bit-wide data stream - channel. When the connection is established, the NNTP server host - MUST send a greeting. The client host and server host then exchange - commands and responses (respectively) until the connection is closed - or aborted. If the connection used is TCP, then the server host - starts the NNTP service by listening on a TCP port. When a client - host wishes to make use of the service, it MUST establish a TCP - connection with the server host by connecting to that host on the - same port on which the server is listening. - - The character set for all NNTP commands is UTF-8 [RFC3629]. Commands - in NNTP MUST consist of a keyword, which MAY be followed by one or - more arguments. A CRLF pair MUST terminate all commands. Multiple - commands MUST NOT be on the same line. Unless otherwise noted - elsewhere in this document, arguments SHOULD consist of printable US- - ASCII characters. Keywords and arguments MUST each be separated by - one or more space or TAB characters. Command lines MUST NOT exceed - 512 octets, which includes the terminating CRLF pair. The arguments - MUST NOT exceed 497 octets. A server MAY relax these limits for - commands defined in an extension. - - Where this specification permits UTF-8 characters outside the range - of U+0000 to U+007F, implementations MUST NOT use the Byte Order Mark - (U+FEFF, encoding %xEF.BB.BF) and MUST use the Word Joiner (U+2060, - encoding %xE2.91.A0) for the meaning Zero Width No-Break Space in - - - -Feather Standards Track [Page 6] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - command lines and the initial lines of responses. Implementations - SHOULD apply these same principles throughout. - - The term "character" means a single Unicode code point. - Implementations are not required to carry out Unicode normalisation. - Thus, U+0084 (A-dieresis) is one character, while U+0041 U+0308 (A - composed with dieresis) is two; the two need not be treated as - equivalent. - - Commands may have variants; if so, they use a second keyword - immediately after the first to indicate which variant is required. - The only such commands in this specification are LIST and MODE. Note - that such variants are sometimes referred to as if they were commands - in their own right: "the LIST ACTIVE" command should be read as - shorthand for "the ACTIVE variant of the LIST command". - - Keywords are case insensitive; the case of keywords for commands MUST - be ignored by the server. Command and response arguments are case or - language specific only when stated, either in this document or in - other relevant specifications. - - In some cases, a command involves more data than just a single line. - The further data may be sent either immediately after the command - line (there are no instances of this in this specification, but there - are in extensions such as [NNTP-STREAM]) or following a request from - the server (indicated by a 3xx response). - - Each response MUST start with a three-digit response code that is - sufficient to distinguish all responses. Certain valid responses are - defined to be multi-line; for all others, the response is contained - in a single line. The initial line of the response MUST NOT exceed - 512 octets, which includes the response code and the terminating CRLF - pair; an extension MAY specify a greater maximum for commands that it - defines, but not for any other command. Single-line responses - consist of an initial line only. Multi-line responses consist of an - initial line followed by a multi-line data block. - - An NNTP server MAY have an inactivity autologout timer. Such a timer - SHOULD be of at least three minutes' duration, with the exception - that there MAY be a shorter limit on how long the server is willing - to wait for the first command from the client. The receipt of any - command from the client during the timer interval SHOULD suffice to - reset the autologout timer. Similarly, the receipt of any - significant amount of data from a client that is sending a multi-line - data block (such as during a POST or IHAVE command) SHOULD suffice to - reset the autologout timer. When the timer expires, the server - SHOULD close the connection without sending any response to the - client. - - - -Feather Standards Track [Page 7] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -3.1.1. Multi-line Data Blocks - - A multi-line data block is used in certain commands and responses. - It MUST adhere to the following rules: - - 1. The block consists of a sequence of zero or more "lines", each - being a stream of octets ending with a CRLF pair. Apart from - those line endings, the stream MUST NOT include the octets NUL, - LF, or CR. - - 2. In a multi-line response, the block immediately follows the CRLF - at the end of the initial line of the response. When used in any - other context, the specific command will define when the block is - sent. - - 3. If any line of the data block begins with the "termination octet" - ("." or %x2E), that line MUST be "dot-stuffed" by prepending an - additional termination octet to that line of the block. - - 4. The lines of the block MUST be followed by a terminating line - consisting of a single termination octet followed by a CRLF pair - in the normal way. Thus, unless it is empty, a multi-line block - is always terminated with the five octets CRLF "." CRLF - (%x0D.0A.2E.0D.0A). - - 5. When a multi-line block is interpreted, the "dot-stuffing" MUST - be undone; i.e., the recipient MUST ensure that, in any line - beginning with the termination octet followed by octets other - than a CRLF pair, that initial termination octet is disregarded. - - 6. Likewise, the terminating line ("." CRLF or %x2E.0D.0A) MUST NOT - be considered part of the multi-line block; i.e., the recipient - MUST ensure that any line beginning with the termination octet - followed immediately by a CRLF pair is disregarded. (The first - CRLF pair of the terminating CRLF "." CRLF of a non-empty block - is, of course, part of the last line of the block.) - - Note that texts using an encoding (such as UTF-16 or UTF-32) that may - contain the octets NUL, LF, or CR other than a CRLF pair cannot be - reliably conveyed in the above format (that is, they violate the MUST - requirement above). However, except when stated otherwise, this - specification does not require the content to be UTF-8, and therefore - (subject to that same requirement) it MAY include octets above and - below 128 mixed arbitrarily. - - This document does not place any limit on the length of a line in a - multi-line block. However, the standards that define the format of - articles may do so. - - - -Feather Standards Track [Page 8] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -3.2. Response Codes - - Each response MUST begin with a three-digit status indicator. These - are status reports from the server and indicate the response to the - last command received from the client. - - The first digit of the response broadly indicates the success, - failure, or progress of the previous command: - - 1xx - Informative message - 2xx - Command completed OK - 3xx - Command OK so far; send the rest of it - 4xx - Command was syntactically correct but failed for some reason - 5xx - Command unknown, unsupported, unavailable, or syntax error - - The next digit in the code indicates the function response category: - - x0x - Connection, setup, and miscellaneous messages - x1x - Newsgroup selection - x2x - Article selection - x3x - Distribution functions - x4x - Posting - x8x - Reserved for authentication and privacy extensions - x9x - Reserved for private use (non-standard extensions) - - Certain responses contain arguments such as numbers and names in - addition to the status indicator. In those cases, to simplify - interpretation by the client, the number and type of such arguments - is fixed for each response code, as is whether the code is - single-line or multi-line. Any extension MUST follow this principle - as well. Note that, for historical reasons, the 211 response code is - an exception to this in that the response may be single-line or - multi-line depending on the command (GROUP or LISTGROUP) that - generated it. In all other cases, the client MUST only use the - status indicator itself to determine the nature of the response. The - exact response codes that can be returned by any given command are - detailed in the description of that command. - - Arguments MUST be separated from the numeric status indicator and - from each other by a single space. All numeric arguments MUST be in - base 10 (decimal) format and MAY have leading zeros. String - arguments MUST contain at least one character and MUST NOT contain - TAB, LF, CR, or space. The server MAY add any text after the - response code or last argument, as appropriate, and the client MUST - NOT make decisions based on this text. Such text MUST be separated - from the numeric status indicator or the last argument by at least - one space. - - - - -Feather Standards Track [Page 9] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The server MUST respond to any command with the appropriate generic - response (given in Section 3.2.1) if it represents the situation. - Otherwise, each recognized command MUST return one of the response - codes specifically listed in its description or in an extension. A - server MAY provide extensions to this specification, including new - commands, new variants or features of existing commands, and other - ways of changing the internal state of the server. However, the - server MUST NOT produce any other responses to a client that does not - invoke any of the additional features. (Therefore, a client that - restricts itself to this specification will only receive the - responses that are listed.) - - If a client receives an unexpected response, it SHOULD use the first - digit of the response to determine the result. For example, an - unexpected 2xx should be taken as success, and an unexpected 4xx or - 5xx as failure. - - Response codes not specified in this document MAY be used for any - installation-specific additional commands also not specified. These - SHOULD be chosen to fit the pattern of x9x specified above. - - Neither this document nor any registered extension (see - Section 3.3.3) will specify any response codes of the x9x pattern. - (Implementers of extensions are accordingly cautioned not to use such - responses for extensions that may subsequently be submitted for - registration.) - -3.2.1. Generic Response Codes - - The server MUST respond to any command with the appropriate one of - the following generic responses if it represents the situation. - - If the command is not recognized, or if it is an optional command - that is not implemented by the server, the response code 500 MUST be - returned. - - If there is a syntax error in the arguments of a recognized command, - including the case where more arguments are provided than the command - specifies or the command line is longer than the server accepts, the - response code 501 MUST be returned. The line MUST NOT be truncated - or split and then interpreted. Note that where a command has - variants depending on a second keyword (e.g., LIST ACTIVE and LIST - NEWSGROUPS), 501 MUST be used when the base command is implemented - but the requested variant is not, and 500 MUST be used only when the - base command itself is not implemented. - - If an argument is required to be a base64-encoded string [RFC4648] - (there are no such arguments in this specification, but there may be - - - -Feather Standards Track [Page 10] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - in extensions) and is not validly encoded, the response code 504 MUST - be returned. - - If the server experiences an internal fault or problem that means it - is unable to carry out the command (for example, a necessary file is - missing or a necessary service could not be contacted), the response - code 403 MUST be returned. If the server recognizes the command but - does not provide an optional feature (for example, because it does - not store the required information), or if it only handles a subset - of legitimate cases (see the HDR command, Section 8.5, for an - example), the response code 503 MUST be returned. - - If the client is not authorized to use the specified facility when - the server is in its current state, then the appropriate one of the - following response codes MUST be used. - - 502: It is necessary to terminate the connection and to start a new - one with the appropriate authority before the command can be used. - Historically, some mode-switching servers (see Section 3.4.1) used - this response to indicate that this command will become available - after the MODE READER command (Section 5.3) is used, but this - usage does not conform to this specification and MUST NOT be used. - Note that the server MUST NOT close the connection immediately - after a 502 response except at the initial connection - (Section 5.1) and with the MODE READER command. - - 480: The client must authenticate itself to the server (that is, it - must provide information as to the identity of the client) before - the facility can be used on this connection. This will involve - the use of an authentication extension such as [NNTP-AUTH]. - - 483: The client must negotiate appropriate privacy protection on the - connection. This will involve the use of a privacy extension such - as [NNTP-TLS]. - - 401: The client must change the state of the connection in some other - manner. The first argument of the response MUST be the capability - label (see Section 5.2) of the facility that provides the - necessary mechanism (usually an extension, which may be a private - extension). The server MUST NOT use this response code except as - specified by the definition of the capability in question. - - If the server has to terminate the connection for some reason, it - MUST give a 400 response code to the next command and then - immediately close the connection. Following a 400 response, clients - SHOULD NOT simply reconnect immediately and retry the same actions. - Rather, a client SHOULD either use an exponentially increasing delay - between retries (e.g., double the waiting time after each 400 - - - -Feather Standards Track [Page 11] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - response) or present any associated text to the user for them to - decide whether and when to retry. - - The client MUST be prepared to receive any of these responses for any - command (except, of course, that the server MUST NOT generate a 500 - response code for mandatory commands). - -3.2.1.1. Examples - - Example of an unknown command: - - [C] MAIL - [S] 500 Unknown command - - Example of an unsupported command: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] NEWNEWS - [S] LIST ACTIVE NEWSGROUPS - [S] . - [C] OVER - [S] 500 Unknown command - - Example of an unsupported variant: - - [C] MODE POSTER - [S] 501 Unknown MODE option - - Example of a syntax error: - - [C] ARTICLE a.message.id@no.angle.brackets - [S] 501 Syntax error - - Example of an overlong command line: - - [C] HEAD 53 54 55 - [S] 501 Too many arguments - - Example of a bad wildmat: - - [C] LIST ACTIVE u[ks].* - [S] 501 Syntax error - - - - - - -Feather Standards Track [Page 12] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of a base64-encoding error (the second argument is meant to - be base64 encoded): - - [C] XENCRYPT RSA abcd=efg - [S] 504 Base64 encoding error - - Example of an attempt to access a facility not available to this - connection: - - [C] MODE READER - [S] 200 Reader mode, posting permitted - [C] IHAVE - [S] 500 Permission denied - - Example of an attempt to access a facility requiring authentication: - - [C] GROUP secret.group - [S] 480 Permission denied - - Example of a successful attempt following such authentication: - - [C] XSECRET fred flintstone - [S] 290 Password for fred accepted - [C] GROUP secret.group - [S] 211 5 1 20 secret.group selected - - Example of an attempt to access a facility requiring privacy: - - [C] GROUP secret.group - [S] 483 Secure connection required - [C] XENCRYPT - [Client and server negotiate encryption on the link] - [S] 283 Encrypted link established - [C] GROUP secret.group - [S] 211 5 1 20 secret.group selected - - Example of a need to change mode before a facility is used: - - [C] GROUP binary.group - [S] 401 XHOST Not on this virtual host - [C] XHOST binary.news.example.org - [S] 290 binary.news.example.org virtual host selected - [C] GROUP binary.group - [S] 211 5 1 77 binary.group selected - - - - - - - -Feather Standards Track [Page 13] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of a temporary failure: - - [C] GROUP archive.local - [S] 403 Archive server temporarily offline - - Example of the server needing to close down immediately: - - [C] ARTICLE 123 - [S] 400 Power supply failed, running on UPS - [Server closes connection.] - -3.3. Capabilities and Extensions - - Not all NNTP servers provide exactly the same facilities, both - because this specification allows variation and because servers may - provide extensions. A set of facilities that are related are called - a "capability". This specification provides a way to determine what - capabilities are available, includes a list of standard capabilities, - and includes a mechanism (the extension mechanism) for defining new - capabilities. - -3.3.1. Capability Descriptions - - A client can determine the available capabilities of the server by - using the CAPABILITIES command (Section 5.2). This returns a - capability list, which is a list of capability lines. Each line - describes one available capability. - - Each capability line consists of one or more tokens, which MUST be - separated by one or more space or TAB characters. A token is a - string of 1 or more printable UTF-8 characters (that is, either - printable US-ASCII characters or any UTF-8 sequence outside the US- - ASCII range, but not space or TAB). Unless stated otherwise, tokens - are case insensitive. Each capability line consists of the - following: - - o The capability label, which is a keyword indicating the - capability. A capability label may be defined by this - specification or a successor, or by an extension. - - o The label is then followed by zero or more tokens, which are - arguments of the capability. The form and meaning of these tokens - is specific to each capability. - - The server MUST ensure that the capability list accurately reflects - the capabilities (including extensions) currently available. If a - capability is only available with the server in a certain state (for - example, only after authentication), the list MUST only include the - - - -Feather Standards Track [Page 14] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - capability label when the server is in that state. Similarly, if - only some of the commands in an extension will be available, or if - the behaviour of the extension will change in some other manner, - according to the state of the server, this MUST be indicated by - different arguments in the capability line. - - Note that a capability line can only begin with a letter. Lines - beginning with other characters are reserved for future versions of - this specification. In order to interoperate with such versions, - clients MUST be prepared to receive lines beginning with other - characters and MUST ignore any they do not understand. - -3.3.2. Standard Capabilities - - The following capabilities are defined by this specification. - - VERSION - This capability MUST be advertised by all servers and MUST be the - first capability in the capability list; it indicates the - version(s) of NNTP that the server supports. There must be at - least one argument; each argument is a decimal number and MUST NOT - have a leading zero. Version numbers are assigned only in RFCs - that update or replace this specification; servers MUST NOT create - their own version numbers. - - The version number of this specification is 2. - - READER - This capability indicates that the server implements the various - commands useful for reading clients. - - IHAVE - This capability indicates that the server implements the IHAVE - command. - - POST - This capability indicates that the server implements the POST - command. - - NEWNEWS - This capability indicates that the server implements the NEWNEWS - command. - - HDR - This capability indicates that the server implements the header - access commands (HDR and LIST HEADERS). - - - - - -Feather Standards Track [Page 15] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - OVER - This capability indicates that the server implements the overview - access commands (OVER and LIST OVERVIEW.FMT). If and only if the - server supports the message-id form of the OVER command, there - must be a single argument MSGID. - - LIST - This capability indicates that the server implements at least one - variant of the LIST command. There MUST be one argument for each - variant of the LIST command supported by the server, giving the - keyword for that variant. - - IMPLEMENTATION - This capability MAY be provided by a server. If so, the arguments - SHOULD be used to provide information such as the server software - name and version number. The client MUST NOT use this line to - determine capabilities of the server. (While servers often - provide this information in the initial greeting, clients need to - guess whether this is the case; this capability makes it clear - what the information is.) - - MODE-READER - This capability indicates that the server is mode-switching - (Section 3.4.2) and that the MODE READER command needs to be used - to enable the READER capability. - -3.3.3. Extensions - - Although NNTP is widely and robustly deployed, some parts of the - Internet community might wish to extend the NNTP service. It must be - emphasized that any extension to NNTP should not be considered - lightly. NNTP's strength comes primarily from its simplicity. - Experience with many protocols has shown that: - - Protocols with few options tend towards ubiquity, whilst protocols - with many options tend towards obscurity. - - This means that each and every extension, regardless of its benefits, - must be carefully scrutinized with respect to its implementation, - deployment, and interoperability costs. In many cases, the cost of - extending the NNTP service will likely outweigh the benefit. - - An extension is a package of associated facilities, often but not - always including one or more new commands. Each extension MUST - define at least one new capability label (this will often, but need - not, be the name of one of these new commands). While any additional - capability information can normally be specified using arguments to - - - - -Feather Standards Track [Page 16] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - that label, an extension MAY define more than one capability label. - However, this SHOULD be limited to exceptional circumstances. - - An extension is either a private extension, or its capabilities are - included in the IANA registry of capabilities (see Section 3.3.4) and - it is defined in an RFC (in which case it is a "registered - extension"). Such RFCs either must be on the standards track or must - define an IESG-approved experimental protocol. - - The definition of an extension must include the following: - - o a descriptive name for the extension. - - o the capability label or labels defined by the extension (the - capability label of a registered extension MUST NOT begin with - "X"). - - o The syntax, values, and meanings of any arguments for each - capability label defined by the extension. - - o Any new NNTP commands associated with the extension (the names of - commands associated with registered extensions MUST NOT begin with - "X"). - - o The syntax and possible values of arguments associated with the - new NNTP commands. - - o The response codes and possible values of arguments for the - responses of the new NNTP commands. - - o Any new arguments the extension associates with any other - pre-existing NNTP commands. - - o Any increase in the maximum length of commands and initial - response lines over the value specified in this document. - - o A specific statement about the effect on pipelining that this - extension may have (if any). - - o A specific statement about the circumstances when use of this - extension can alter the contents of the capabilities list (other - than the new capability labels it defines). - - o A specific statement about the circumstances under which the - extension can cause any pre-existing command to produce a 401, - 480, or 483 response. - - - - - -Feather Standards Track [Page 17] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - o A description of how the use of MODE READER on a mode-switching - server interacts with the extension. - - o A description of how support for the extension affects the - behaviour of a server and NNTP client in any other manner not - outlined above. - - o Formal syntax as described in Section 9.9. - - A private extension MAY or MAY NOT be included in the capabilities - list. If it is, the capability label MUST begin with "X". A server - MAY provide additional keywords (for new commands and also for new - variants of existing commands) as part of a private extension. To - avoid the risk of a clash with a future registered extension, these - keywords SHOULD begin with "X". - - If the server advertises a capability defined by a registered - extension, it MUST implement the extension so as to fully conform - with the specification (for example, it MUST implement all the - commands that the extension describes as mandatory). If it does not - implement the extension as specified, it MUST NOT list the extension - in the capabilities list under its registered name. In that case, it - MAY, but SHOULD NOT, provide a private extension (not listed, or - listed with a different name) that implements part of the extension - or implements the commands of the extension with a different meaning. - - A server MUST NOT send different response codes to basic NNTP - commands documented here or to commands documented in registered - extensions in response to the availability or use of a private - extension. - -3.3.4. Initial IANA Register - - IANA will maintain a registry of NNTP capability labels. All - capability labels in the registry MUST be keywords and MUST NOT begin - with X. - - - - - - - - - - - - - - - -Feather Standards Track [Page 18] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The initial content of the registry consists of these entries: - - +-------------------+--------------------------+--------------------+ - | Label | Meaning | Definition | - +-------------------+--------------------------+--------------------+ - | AUTHINFO | Authentication | [NNTP-AUTH] | - | | | | - | HDR | Batched header retrieval | Section 3.3.2, | - | | | Section 8.5, and | - | | | Section 8.6 | - | | | | - | IHAVE | IHAVE command available | Section 3.3.2 and | - | | | Section 6.3.2 | - | | | | - | IMPLEMENTATION | Server | Section 3.3.2 | - | | implementation-specific | | - | | information | | - | | | | - | LIST | LIST command variants | Section 3.3.2 and | - | | | Section 7.6.1 | - | | | | - | MODE-READER | Mode-switching server | Section 3.4.2 | - | | and MODE READER command | | - | | available | | - | | | | - | NEWNEWS | NEWNEWS command | Section 3.3.2 and | - | | available | Section 7.4 | - | | | | - | OVER | Overview support | Section 3.3.2, | - | | | Section 8.3, and | - | | | Section 8.4 | - | | | | - | POST | POST command available | Section 3.3.2 and | - | | | Section 6.3.1 | - | | | | - | READER | Reader commands | Section 3.3.2 | - | | available | | - | | | | - | SASL | Supported SASL | [NNTP-AUTH] | - | | mechanisms | | - | | | | - | STARTTLS | Transport layer security | [NNTP-TLS] | - | | | | - | STREAMING | Streaming feeds | [NNTP-STREAM] | - | | | | - | VERSION | Supported NNTP versions | Section 3.3.2 | - +-------------------+--------------------------+--------------------+ - - - - -Feather Standards Track [Page 19] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -3.4. Mandatory and Optional Commands - - For a number of reasons, not all the commands in this specification - are mandatory. However, it is equally undesirable for every command - to be optional, since this means that a client will have no idea what - facilities are available. Therefore, as a compromise, some of the - commands in this specification are mandatory (they must be supported - by all servers) while the remainder are not. The latter are then - subdivided into bundles, each indicated by a single capability label. - - o If the label is included in the capability list returned by the - server, the server MUST support all commands in that bundle. - - o If the label is not included, the server MAY support none or some - of the commands but SHOULD NOT support all of them. In general, - there will be no way for a client to determine which commands are - supported without trying them. - - The bundles have been chosen to provide useful functionality, and - therefore server authors are discouraged from implementing only part - of a bundle. - - The description of each command will either indicate that it is - mandatory, or will give, using the term "indicating capability", the - capability label indicating whether the bundle including this command - is available. - - Where a server does not implement a command, it MUST always generate - a 500 generic response code (or a 501 generic response code in the - case of a variant of a command depending on a second keyword where - the base command is recognised). Otherwise, the command MUST be - fully implemented as specified; a server MUST NOT only partially - implement any of the commands in this specification. (Client authors - should note that some servers not conforming to this specification - will return a 502 generic response code to some commands that are not - implemented.) - - Note: some commands have cases that require other commands to be used - first. If the former command is implemented but the latter is not, - the former MUST still generate the relevant specific response code. - For example, if ARTICLE (Section 6.2.1) is implemented but GROUP - (Section 6.1.1) is not, the correct response to "ARTICLE 1234" - remains 412. - - - - - - - - -Feather Standards Track [Page 20] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -3.4.1. Reading and Transit Servers - - NNTP is traditionally used in two different ways. The first use is - "reading", where the client fetches articles from a large store - maintained by the server for immediate or later presentation to a - user and sends articles created by that user back to the server (an - action called "posting") to be stored and distributed to other stores - and users. The second use is for the bulk transfer of articles from - one store to another. Since the hosts making this transfer tend to - be peers in a network that transmit articles among one another, and - not end-user systems, this process is called "peering" or "transit". - (Even so, one host is still the client and the other is the server). - - In practice, these two uses are so different that some server - implementations are optimised for reading or for transit and, as a - result, do not offer the other facility or only offer limited - features. Other implementations are more general and offer both. - This specification allows for this by bundling the relevant commands - accordingly: the IHAVE command is designed for transit, while the - commands indicated by the READER capability are designed for reading - clients. - - Except as an effect of the MODE READER command (Section 5.3) on a - mode-switching server, once a server advertises either or both of the - IHAVE or READER capabilities, it MUST continue to advertise them for - the entire session. - - A server MAY provide different modes of behaviour (transit, reader, - or a combination) to different client connections and MAY use - external information, such as the IP address of the client, to - determine which mode to provide to any given connection. - - The official TCP port for the NNTP service is 119. However, if a - host wishes to offer separate servers for transit and reading - clients, port 433 SHOULD be used for the transit server and 119 for - the reading server. - -3.4.2. Mode Switching - - An implementation MAY, but SHOULD NOT, provide both transit and - reader facilities on the same server but require the client to select - which it wishes to use. Such an arrangement is called a - "mode-switching" server. - - - - - - - - -Feather Standards Track [Page 21] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - A mode-switching server has two modes: - - o Transit mode, which applies after the initial connection. - - * It MUST advertise the MODE-READER capability. - - * It MUST NOT advertise the READER capability. - - However, the server MAY cease to advertise the MODE-READER - capability after the client uses any command except CAPABILITIES. - - o Reading mode, after a successful MODE READER command (see Section - 5.3). - - * It MUST NOT advertise the MODE-READER capability. - - * It MUST advertise the READER capability. - - * It MAY NOT advertise the IHAVE capability, even if it was - advertising it in transit mode. - - A client SHOULD only issue a MODE READER command to a server if it is - advertising the MODE-READER capability. If the server does not - support CAPABILITIES (and therefore does not conform to this - specification), the client MAY use the following heuristic: - - o If the client wishes to use any "reader" commands, it SHOULD use - the MODE READER command immediately after the initial connection. - - o Otherwise, it SHOULD NOT use the MODE READER command. - - In each case, it should be prepared for some commands to be - unavailable that would have been available if it had made the other - choice. - -3.5. Pipelining - - NNTP is designed to operate over a reliable bi-directional - connection, such as TCP. Therefore, if a command does not depend on - the response to the previous one, it should not matter if it is sent - before that response is received. Doing this is called "pipelining". - However, certain server implementations throw away all text received - from the client following certain commands before sending their - response. If this happens, pipelining will be affected because one - or more commands will have been ignored or misinterpreted, and the - client will be matching the wrong responses to each command. Since - there are significant benefits to pipelining, but also circumstances - where it is reasonable or common for servers to behave in the above - - - -Feather Standards Track [Page 22] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - manner, this document puts certain requirements on both clients and - servers. - - Except where stated otherwise, a client MAY use pipelining. That is, - it may send a command before receiving the response for the previous - command. The server MUST allow pipelining and MUST NOT throw away - any text received after a command. Irrespective of whether - pipelining is used, the server MUST process commands in the order - they are sent. - - If the specific description of a command says it "MUST NOT be - pipelined", that command MUST end any pipeline of commands. That is, - the client MUST NOT send any following command until it receives the - CRLF at the end of the response from the command. The server MAY - ignore any data received after the command and before the CRLF at the - end of the response is sent to the client. - - The initial connection must not be part of a pipeline; that is, the - client MUST NOT send any command until it receives the CRLF at the - end of the greeting. - - If the client uses blocking system calls to send commands, it MUST - ensure that the amount of text sent in pipelining does not cause a - deadlock between transmission and reception. The amount of text - involved will depend on window sizes in the transmission layer; - typically, it is 4k octets for TCP. (Since the server only sends - data in response to commands from the client, the converse problem - does not occur.) - -3.5.1. Examples - - Example of correct use of pipelining: - - [C] GROUP misc.test - [C] STAT - [C] NEXT - [S] 211 1234 3000234 3002322 misc.test - [S] 223 3000234 <45223423@example.com> retrieved - [S] 223 3000237 <668929@example.org> retrieved - - Example of incorrect use of pipelining (the MODE READER command may - not be pipelined): - - [C] MODE READER - [C] DATE - [C] NEXT - [S] 200 Server ready, posting allowed - [S] 223 3000237 <668929@example.org> retrieved - - - -Feather Standards Track [Page 23] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The DATE command has been thrown away by the server, so there is no - 111 response to match it. - -3.6. Articles - - NNTP is intended to transfer articles between clients and servers. - For the purposes of this specification, articles are required to - conform to the rules in this section, and clients and servers MUST - correctly process any article received from the other that does so. - Note that this requirement applies only to the contents of - communications over NNTP; it does not prevent the client or server - from subsequently rejecting an article for reasons of local policy. - Also see Appendix A for further restrictions on the format of - articles in some uses of NNTP. - - An article consists of two parts: the headers and the body. They are - separated by a single empty line, or in other words by two - consecutive CRLF pairs (if there is more than one empty line, the - second and subsequent ones are part of the body). In order to meet - the general requirements of NNTP, an article MUST NOT include the - octet NUL, MUST NOT contain the octets LF and CR other than as part - of a CRLF pair, and MUST end with a CRLF pair. This specification - puts no further restrictions on the body; in particular, it MAY be - empty. - - The headers of an article consist of one or more header lines. Each - header line consists of a header name, a colon, a space, the header - content, and a CRLF, in that order. The name consists of one or more - printable US-ASCII characters other than colon and, for the purposes - of this specification, is not case sensitive. There MAY be more than - one header line with the same name. The content MUST NOT contain - CRLF; it MAY be empty. A header may be "folded"; that is, a CRLF - pair may be placed before any TAB or space in the line. There MUST - still be some other octet between any two CRLF pairs in a header - line. (Note that folding means that the header line occupies more - than one line when displayed or transmitted; nevertheless, it is - still referred to as "a" header line.) The presence or absence of - folding does not affect the meaning of the header line; that is, the - CRLF pairs introduced by folding are not considered part of the - header content. Header lines SHOULD NOT be folded before the space - after the colon that follows the header name and SHOULD include at - least one octet other than %x09 or %x20 between CRLF pairs. However, - if an article that fails to satisfy this requirement has been - received from elsewhere, clients and servers MAY transfer it to each - other without re-folding it. - - - - - - -Feather Standards Track [Page 24] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The content of a header SHOULD be in UTF-8. However, if an - implementation receives an article from elsewhere that uses octets in - the range 128 to 255 in some other manner, it MAY pass it to a client - or server without modification. Therefore, implementations MUST be - prepared to receive such headers, and data derived from them (e.g., - in the responses from the OVER command, Section 8.3), and MUST NOT - assume that they are always UTF-8. Any external processing of those - headers, including identifying the encoding used, is outside the - scope of this document. - - Each article MUST have a unique message-id; two articles offered by - an NNTP server MUST NOT have the same message-id. For the purposes - of this specification, message-ids are opaque strings that MUST meet - the following requirements: - - o A message-id MUST begin with "<", end with ">", and MUST NOT - contain the latter except at the end. - - o A message-id MUST be between 3 and 250 octets in length. - - o A message-id MUST NOT contain octets other than printable US-ASCII - characters. - - Two message-ids are the same if and only if they consist of the same - sequence of octets. - - This specification does not describe how the message-id of an article - is determined. If the server does not have any way to determine a - message-id from the article itself, it MUST synthesize one (this - specification does not require that the article be changed as a - result). See also Appendix A.2. - -4. The WILDMAT Format - - The WILDMAT format described here is based on the version first - developed by Rich Salz [SALZ1992], which was in turn derived from the - format used in the UNIX "find" command to articulate file names. It - was developed to provide a uniform mechanism for matching patterns in - the same manner that the UNIX shell matches filenames. - - - - - - - - - - - - -Feather Standards Track [Page 25] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -4.1. Wildmat Syntax - - A wildmat is described by the following ABNF [RFC4234] syntax, which - is an extract of that in Section 9.8. - - wildmat = wildmat-pattern *("," ["!"] wildmat-pattern) - wildmat-pattern = 1*wildmat-item - wildmat-item = wildmat-exact / wildmat-wild - wildmat-exact = %x22-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E / - UTF8-non-ascii ; exclude ! * , ? [ \ ] - wildmat-wild = "*" / "?" - - Note: the characters ",", "\", "[", and "]" are not allowed in - wildmats, while * and ? are always wildcards. This should not be a - problem, since these characters cannot occur in newsgroup names, - which is the only current use of wildmats. Backslash is commonly - used to suppress the special meaning of characters, whereas brackets - are used to introduce sets. However, these usages are not universal, - and interpretation of these characters in the context of UTF-8 - strings is potentially complex and differs from existing practice, so - they were omitted from this specification. A future extension to - this specification may provide semantics for these characters. - -4.2. Wildmat Semantics - - A wildmat is tested against a string and either matches or does not - match. To do this, each constituent is matched - against the string, and the rightmost pattern that matches is - identified. If that is not preceded with "!", the - whole wildmat matches. If it is preceded by "!", or if no matches, the whole wildmat does not match. - - For example, consider the wildmat "a*,!*b,*c*": - - o The string "aaa" matches because the rightmost match is with "a*". - - o The string "abb" does not match because the rightmost match is - with "*b". - - o The string "ccb" matches because the rightmost match is with - "*c*". - - o The string "xxx" does not match because no - matches. - - A matches a string if the string can be broken into - components, each of which matches the corresponding in - the pattern. The matches must be in the same order, and the whole - - - -Feather Standards Track [Page 26] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - string must be used in the match. The pattern is "anchored"; that - is, the first and last characters in the string must match the first - and last item, respectively (unless that item is an asterisk matching - zero characters). - - A matches the same character (which may be more than - one octet in UTF-8). - - "?" matches exactly one character (which may be more than one octet). - - "*" matches zero or more characters. It can match an empty string, - but it cannot match a subsequence of a UTF-8 sequence that is not - aligned to the character boundaries. - -4.3. Extensions - - An NNTP server or extension MAY extend the syntax or semantics of - wildmats provided that all wildmats that meet the requirements of - Section 4.1 have the meaning ascribed to them by Section 4.2. Future - editions of this document may also extend wildmats. - -4.4. Examples - - In these examples, $ and @ are used to represent the two octets %xC2 - and %xA3, respectively; $@ is thus the UTF-8 encoding for the pound - sterling symbol, shown as # in the descriptions. - - Wildmat Description of strings that match - abc The one string "abc" - abc,def The two strings "abc" and "def" - $@ The one character string "#" - a* Any string that begins with "a" - a*b Any string that begins with "a" and ends with "b" - a*,*b Any string that begins with "a" or ends with "b" - a*,!*b Any string that begins with "a" and does not end with - "b" - a*,!*b,c* Any string that begins with "a" and does not end with - "b", and any string that begins with "c" no matter - what it ends with - a*,c*,!*b Any string that begins with "a" or "c" and does not - end with "b" - ?a* Any string with "a" as its second character - ??a* Any string with "a" as its third character - *a? Any string with "a" as its penultimate character - *a?? Any string with "a" as its antepenultimate character - - - - - - -Feather Standards Track [Page 27] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -5. Session Administration Commands - -5.1. Initial Connection - -5.1.1. Usage - - This command MUST NOT be pipelined. - - Responses [1] - 200 Service available, posting allowed - 201 Service available, posting prohibited - 400 Service temporarily unavailable [2] - 502 Service permanently unavailable [2] - - [1] These are the only valid response codes for the initial greeting; - the server MUST not return any other generic response code. - - [2] Following a 400 or 502 response, the server MUST immediately - close the connection. - -5.1.2. Description - - There is no command presented by the client upon initial connection - to the server. The server MUST present an appropriate response code - as a greeting to the client. This response informs the client - whether service is available and whether the client is permitted to - post. - - If the server will accept further commands from the client including - POST, the server MUST present a 200 greeting code. If the server - will accept further commands from the client, but the client is not - authorized to post articles using the POST command, the server MUST - present a 201 greeting code. - - Otherwise, the server MUST present a 400 or 502 greeting code and - then immediately close the connection. 400 SHOULD be used if the - issue is only temporary (for example, because of load) and the client - can expect to be able to connect successfully at some point in the - future without making any changes. 502 MUST be used if the client is - not permitted under any circumstances to interact with the server, - and MAY be used if the server has insufficient information to - determine whether the issue is temporary or permanent. - - Note: the distinction between the 200 and 201 response codes has - turned out in practice to be insufficient; for example, some servers - do not allow posting until the client has authenticated, while other - clients assume that a 201 response means that posting will never be - possible even after authentication. Therefore, clients SHOULD use - - - -Feather Standards Track [Page 28] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - the CAPABILITIES command (Section 5.2) rather than rely on this - response. - -5.1.3. Examples - - Example of a normal connection from an authorized client that then - terminates the session (see Section 5.4): - - [Initial connection set-up completed.] - [S] 200 NNTP Service Ready, posting permitted - [C] QUIT - [S] 205 NNTP Service exits normally - [Server closes connection.] - - Example of a normal connection from an authorized client that is not - permitted to post, which also immediately terminates the session: - - [Initial connection set-up completed.] - [S] 201 NNTP Service Ready, posting prohibited - [C] QUIT - [S] 205 NNTP Service exits normally - [Server closes connection.] - - Example of a normal connection from an unauthorized client: - - [Initial connection set-up completed.] - [S] 502 NNTP Service permanently unavailable - [Server closes connection.] - - Example of a connection from a client if the server is unable to - provide service: - - [Initial connection set-up completed.] - [S] 400 NNTP Service temporarily unavailable - [Server closes connection.] - -5.2. CAPABILITIES - -5.2.1. Usage - - This command is mandatory. - - Syntax - CAPABILITIES [keyword] - - Responses - 101 Capability list follows (multi-line) - - - - -Feather Standards Track [Page 29] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Parameters - keyword additional feature, see description - -5.2.2. Description - - The CAPABILITIES command allows a client to determine the - capabilities of the server at any given time. - - This command MAY be issued at any time; the server MUST NOT require - it to be issued in order to make use of any capability. The response - generated by this command MAY change during a session because of - other state information (which, in turn, may be changed by the - effects of other commands or by external events). An NNTP client is - only able to get the current and correct information concerning - available capabilities at any point during a session by issuing a - CAPABILITIES command at that point of that session and processing the - response. - - The capability list is returned as a multi-line data block following - the 101 response code. Each capability is described by a separate - capability line. The server MUST NOT list the same capability twice - in the response, even with different arguments. Except that the - VERSION capability MUST be the first line, the order in which the - capability lines appears is not significant; the server need not even - consistently return the same order. - - While some capabilities are likely to be always available or never - available, others (notably extensions) will appear and disappear - depending on server state changes within the session or on external - events between sessions. An NNTP client MAY cache the results of - this command, but MUST NOT rely on the correctness of any cached - results, whether from earlier in this session or from a previous - session, MUST cope gracefully with the cached status being out of - date, and SHOULD (if caching results) provide a way to force the - cached information to be refreshed. Furthermore, a client MUST NOT - use cached results in relation to security, privacy, and - authentication extensions. See Section 12.6 for further discussion - of this topic. - - The keyword argument is not used by this specification. It is - provided so that extensions or revisions to this specification can - include extra features for this command without requiring the - CAPABILITIES command to be used twice (once to determine if the extra - features are available, and a second time to make use of them). If - the server does not recognise the argument (and it is a keyword), it - MUST respond with the 101 response code as if the argument had been - omitted. If an argument is provided that the server does recognise, - it MAY use the 101 response code or MAY use some other response code - - - -Feather Standards Track [Page 30] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - (which will be defined in the specification of that feature). If the - argument is not a keyword, the 501 generic response code MUST be - returned. The server MUST NOT generate any other response code to - the CAPABILITIES command. - -5.2.3. Examples - - Example of a minimal response (a read-only server): - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] LIST ACTIVE NEWSGROUPS - [S] . - - Example of a response from a server that has a range of facilities - and that also describes itself: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] IHAVE - [S] POST - [S] NEWNEWS - [S] LIST ACTIVE NEWSGROUPS ACTIVE.TIMES OVERVIEW.FMT - [S] IMPLEMENTATION INN 4.2 2004-12-25 - [S] OVER MSGID - [S] STREAMING - [S] XSECRET - [S] . - - Example of a server that supports more than one version of NNTP: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 3 - [S] READER - [S] LIST ACTIVE NEWSGROUPS - [S] . - - - - - - - - - - -Feather Standards Track [Page 31] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of a client attempting to use a feature of the CAPABILITIES - command that the server does not support: - - [C] CAPABILITIES AUTOUPDATE - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] IHAVE - [S] LIST ACTIVE NEWSGROUPS OVERVIEW.FMT HEADERS - [S] OVER MSGID - [S] HDR - [S] NEWNEWS - [S] . - -5.3. MODE READER - -5.3.1. Usage - - Indicating capability: MODE-READER - - This command MUST NOT be pipelined. - - Syntax - MODE READER - - Responses - 200 Posting allowed - 201 Posting prohibited - 502 Reading service permanently unavailable [1] - - [1] Following a 502 response the server MUST immediately close the - connection. - -5.3.2. Description - - The MODE READER command instructs a mode-switching server to switch - modes, as described in Section 3.4.2. - - If the server is mode-switching, it switches from its transit mode to - its reader mode, indicating this by changing the capability list - accordingly. It MUST then return a 200 or 201 response with the same - meaning as for the initial greeting (as described in Section 5.1.1). - Note that the response need not be the same as that presented during - the initial greeting. The client MUST NOT issue MODE READER more - than once in a session or after any security or privacy commands are - issued. When the MODE READER command is issued, the server MAY reset - its state to that immediately after the initial connection before - switching mode. - - - -Feather Standards Track [Page 32] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - If the server is not mode-switching, then the following apply: - - o If it advertises the READER capability, it MUST return a 200 or - 201 response with the same meaning as for the initial greeting; in - this case, the command MUST NOT affect the server state in any - way. - - o If it does not advertise the READER capability, it MUST return a - 502 response and then immediately close the connection. - -5.3.3. Examples - - Example of use of the MODE READER command on a transit-only server - (which therefore does not providing reading facilities): - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] IHAVE - [S] . - [C] MODE READER - [S] 502 Transit service only - [Server closes connection.] - - Example of use of the MODE READER command on a server that provides - reading facilities: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] LIST ACTIVE NEWSGROUPS - [S] . - [C] MODE READER - [S] 200 Reader mode, posting permitted - [C] IHAVE - [S] 500 Permission denied - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - - Note that in both of these situations, the client SHOULD NOT use MODE - READER. - - - - - - - - - -Feather Standards Track [Page 33] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of use of the MODE READER command on a mode-switching server: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] IHAVE - [S] MODE-READER - [S] . - [C] MODE READER - [S] 200 Reader mode, posting permitted - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] NEWNEWS - [S] LIST ACTIVE NEWSGROUPS - [S] STARTTLS - [S] . - - In this case, the server offers (but does not require) TLS privacy in - its reading mode but not in its transit mode. - - Example of use of the MODE READER command where the client is not - permitted to post: - - [C] MODE READER - [S] 201 NNTP Service Ready, posting prohibited - -5.4. QUIT - -5.4.1. Usage - - This command is mandatory. - - Syntax - QUIT - - Responses - 205 Connection closing - -5.4.2. Description - - The client uses the QUIT command to terminate the session. The - server MUST acknowledge the QUIT command and then close the - connection to the client. This is the preferred method for a client - to indicate that it has finished all of its transactions with the - NNTP server. - - - - -Feather Standards Track [Page 34] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - If a client simply disconnects (or if the connection times out or - some other fault occurs), the server MUST gracefully cease its - attempts to service the client, disconnecting from its end if - necessary. - - The server MUST NOT generate any response code to the QUIT command - other than 205 or, if any arguments are provided, 501. - -5.4.3. Examples - - [C] QUIT - [S] 205 closing connection - [Server closes connection.] - -6. Article Posting and Retrieval - - News-reading clients have available a variety of mechanisms to - retrieve articles via NNTP. The news articles are stored and indexed - using three types of keys. The first type of key is the message-id - of an article and is globally unique. The second type of key is - composed of a newsgroup name and an article number within that - newsgroup. On a particular server, there MUST only be one article - with a given number within any newsgroup, and an article MUST NOT - have two different numbers in the same newsgroup. An article can be - cross-posted to multiple newsgroups, so there may be multiple keys - that point to the same article on the same server; these MAY have - different numbers in each newsgroup. However, this type of key is - not required to be globally unique, so the same key MAY refer to - different articles on different servers. (Note that the terms - "group" and "newsgroup" are equivalent.) - - The final type of key is the arrival timestamp, giving the time that - the article arrived at the server. The server MUST ensure that - article numbers are issued in order of arrival timestamp; that is, - articles arriving later MUST have higher numbers than those that - arrive earlier. The server SHOULD allocate the next sequential - unused number to each new article. - - Article numbers MUST lie between 1 and 2,147,483,647, inclusive. The - client and server MAY use leading zeroes in specifying article - numbers but MUST NOT use more than 16 digits. In some situations, - the value zero replaces an article number to show some special - situation. - - Note that it is likely that the article number limit of 2,147,483,647 - will be increased by a future revision or extension to this - specification. While servers MUST NOT send article numbers greater - than this current limit, client and server developers are advised to - - - -Feather Standards Track [Page 35] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - use internal structures and datatypes capable of handling larger - values in anticipation of such a change. - -6.1. Group and Article Selection - - The following commands are used to set the "currently selected - newsgroup" and the "current article number", which are used by - various commands. At the start of an NNTP session, both of these - values are set to the special value "invalid". - -6.1.1. GROUP - -6.1.1.1. Usage - - Indicating capability: READER - - Syntax - GROUP group - - Responses - 211 number low high group Group successfully selected - 411 No such newsgroup - - Parameters - group Name of newsgroup - number Estimated number of articles in the group - low Reported low water mark - high Reported high water mark - -6.1.1.2. Description - - The GROUP command selects a newsgroup as the currently selected - newsgroup and returns summary information about it. - - The required argument is the name of the newsgroup to be selected - (e.g., "news.software.nntp"). A list of valid newsgroups may be - obtained by using the LIST ACTIVE command (see Section 7.6.3). - - The successful selection response will return the article numbers of - the first and last articles in the group at the moment of selection - (these numbers are referred to as the "reported low water mark" and - the "reported high water mark") and an estimate of the number of - articles in the group currently available. - - If the group is not empty, the estimate MUST be at least the actual - number of articles available and MUST be no greater than one more - than the difference between the reported low and high water marks. - (Some implementations will actually count the number of articles - - - -Feather Standards Track [Page 36] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - currently stored. Others will just subtract the low water mark from - the high water mark and add one to get an estimate.) - - If the group is empty, one of the following three situations will - occur. Clients MUST accept all three cases; servers MUST NOT - represent an empty group in any other way. - - o The high water mark will be one less than the low water mark, and - the estimated article count will be zero. Servers SHOULD use this - method to show an empty group. This is the only time that the - high water mark can be less than the low water mark. - - o All three numbers will be zero. - - o The high water mark is greater than or equal to the low water - mark. The estimated article count might be zero or non-zero; if - it is non-zero, the same requirements apply as for a non-empty - group. - - The set of articles in a group may change after the GROUP command is - carried out: - - o Articles may be removed from the group. - - o Articles may be reinstated in the group with the same article - number, but those articles MUST have numbers no less than the - reported low water mark (note that this is a reinstatement of the - previous article, not a new article reusing the number). - - o New articles may be added with article numbers greater than the - reported high water mark. (If an article that was the one with - the highest number has been removed and the high water mark has - been adjusted accordingly, the next new article will not have the - number one greater than the reported high water mark.) - - Except when the group is empty and all three numbers are zero, - whenever a subsequent GROUP command for the same newsgroup is issued, - either by the same client or a different client, the reported low - water mark in the response MUST be no less than that in any previous - response for that newsgroup in this session, and it SHOULD be no less - than that in any previous response for that newsgroup ever sent to - any client. Any failure to meet the latter condition SHOULD be - transient only. The client may make use of the low water mark to - remove all remembered information about articles with lower numbers, - as these will never recur. This includes the situation when the high - water mark is one less than the low water mark. No similar - assumption can be made about the high water mark, as this can - - - - -Feather Standards Track [Page 37] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - decrease if an article is removed and then increase again if it is - reinstated or if new articles arrive. - - When a valid group is selected by means of this command, the - currently selected newsgroup MUST be set to that group, and the - current article number MUST be set to the first article in the group - (this applies even if the group is already the currently selected - newsgroup). If an empty newsgroup is selected, the current article - number is made invalid. If an invalid group is specified, the - currently selected newsgroup and current article number MUST NOT be - changed. - - The GROUP or LISTGROUP command (see Section 6.1.2) MUST be used by a - client, and a successful response received, before any other command - is used that depends on the value of the currently selected newsgroup - or current article number. - - If the group specified is not available on the server, a 411 response - MUST be returned. - -6.1.1.3. Examples - - Example for a group known to the server: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - - Example for a group unknown to the server: - - [C] GROUP example.is.sob.bradner.or.barber - [S] 411 example.is.sob.bradner.or.barber is unknown - - Example of an empty group using the preferred response: - - [C] GROUP example.currently.empty.newsgroup - [S] 211 0 4000 3999 example.currently.empty.newsgroup - - Example of an empty group using an alternative response: - - [C] GROUP example.currently.empty.newsgroup - [S] 211 0 0 0 example.currently.empty.newsgroup - - Example of an empty group using a different alternative response: - - [C] GROUP example.currently.empty.newsgroup - [S] 211 0 4000 4321 example.currently.empty.newsgroup - - - - - -Feather Standards Track [Page 38] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example reselecting the currently selected newsgroup: - - [C] GROUP misc.test - [S] 211 1234 234 567 misc.test - [C] STAT 444 - [S] 223 444 <123456@example.net> retrieved - [C] GROUP misc.test - [S] 211 1234 234 567 misc.test - [C] STAT - [S] 223 234 retrieved - -6.1.2. LISTGROUP - -6.1.2.1. Usage - - Indicating capability: READER - - Syntax - LISTGROUP [group [range]] - - Responses - 211 number low high group Article numbers follow (multi-line) - 411 No such newsgroup - 412 No newsgroup selected [1] - - Parameters - group Name of newsgroup - range Range of articles to report - number Estimated number of articles in the group - low Reported low water mark - high Reported high water mark - - [1] The 412 response can only occur if no group has been specified. - -6.1.2.2. Description - - The LISTGROUP command selects a newsgroup in the same manner as the - GROUP command (see Section 6.1.1) but also provides a list of article - numbers in the newsgroup. If no group is specified, the currently - selected newsgroup is used. - - On success, a list of article numbers is returned as a multi-line - data block following the 211 response code (the arguments on the - initial response line are the same as for the GROUP command). The - list contains one number per line and is in numerical order. It - lists precisely those articles that exist in the group at the moment - of selection (therefore, an empty group produces an empty list). If - the optional range argument is specified, only articles within the - - - -Feather Standards Track [Page 39] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - range are included in the list (therefore, the list MAY be empty even - if the group is not). - - The range argument may be any of the following: - - o An article number. - - o An article number followed by a dash to indicate all following. - - o An article number followed by a dash followed by another article - number. - - In the last case, if the second number is less than the first number, - then the range contains no articles. Omitting the range is - equivalent to the range 1- being specified. - - If the group specified is not available on the server, a 411 response - MUST be returned. If no group is specified and the currently - selected newsgroup is invalid, a 412 response MUST be returned. - - Except that the group argument is optional, that a range argument can - be specified, and that a multi-line data block follows the 211 - response code, the LISTGROUP command is identical to the GROUP - command. In particular, when successful, the command sets the - current article number to the first article in the group, if any, - even if this is not within the range specified by the second - argument. - - Note that the range argument is a new feature in this specification - and servers that do not support CAPABILITIES (and therefore do not - conform to this specification) are unlikely to support it. - -6.1.2.3. Examples - - Example of LISTGROUP being used to select a group: - - [C] LISTGROUP misc.test - [S] 211 2000 3000234 3002322 misc.test list follows - [S] 3000234 - [S] 3000237 - [S] 3000238 - [S] 3000239 - [S] 3002322 - [S] . - - - - - - - -Feather Standards Track [Page 40] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of LISTGROUP on an empty group: - - [C] LISTGROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup list follows - [S] . - - Example of LISTGROUP on a valid, currently selected newsgroup: - - [C] GROUP misc.test - [S] 211 2000 3000234 3002322 misc.test - [C] LISTGROUP - [S] 211 2000 3000234 3002322 misc.test list follows - [S] 3000234 - [S] 3000237 - [S] 3000238 - [S] 3000239 - [S] 3002322 - [S] . - - Example of LISTGROUP with a range: - - [C] LISTGROUP misc.test 3000238-3000248 - [S] 211 2000 3000234 3002322 misc.test list follows - [S] 3000238 - [S] 3000239 - [S] . - - Example of LISTGROUP with an empty range: - - [C] LISTGROUP misc.test 12345678- - [S] 211 2000 3000234 3002322 misc.test list follows - [S] . - - Example of LISTGROUP with an invalid range: - - [C] LISTGROUP misc.test 9999-111 - [S] 211 2000 3000234 3002322 misc.test list follows - [S] . - - - - - - - - - - - - - -Feather Standards Track [Page 41] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.1.3. LAST - -6.1.3.1. Usage - - Indicating capability: READER - - Syntax - LAST - - Responses - 223 n message-id Article found - 412 No newsgroup selected - 420 Current article number is invalid - 422 No previous article in this group - - Parameters - n Article number - message-id Article message-id - -6.1.3.2. Description - - If the currently selected newsgroup is valid, the current article - number MUST be set to the previous article in that newsgroup (that - is, the highest existing article number less than the current article - number). If successful, a response indicating the new current - article number and the message-id of that article MUST be returned. - No article text is sent in response to this command. - - There MAY be no previous article in the group, although the current - article number is not the reported low water mark. There MUST NOT be - a previous article when the current article number is the reported - low water mark. - - Because articles can be removed and added, the results of multiple - LAST and NEXT commands MAY not be consistent over the life of a - particular NNTP session. - - If the current article number is already the first article of the - newsgroup, a 422 response MUST be returned. If the current article - number is invalid, a 420 response MUST be returned. If the currently - selected newsgroup is invalid, a 412 response MUST be returned. In - all three cases, the currently selected newsgroup and current article - number MUST NOT be altered. - - - - - - - - -Feather Standards Track [Page 42] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.1.3.3. Examples - - Example of a successful article retrieval using LAST: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] NEXT - [S] 223 3000237 <668929@example.org> retrieved - [C] LAST - [S] 223 3000234 <45223423@example.com> retrieved - - Example of an attempt to retrieve an article without having selected - a group (via the GROUP command) first: - - [Assumes currently selected newsgroup is invalid.] - [C] LAST - [S] 412 no newsgroup selected - - Example of an attempt to retrieve an article using the LAST command - when the current article number is that of the first article in the - group: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] LAST - [S] 422 No previous article to retrieve - - Example of an attempt to retrieve an article using the LAST command - when the currently selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] LAST - [S] 420 No current article selected - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 43] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.1.4. NEXT - -6.1.4.1. Usage - - Indicating capability: READER - - Syntax - NEXT - - Responses - 223 n message-id Article found - 412 No newsgroup selected - 420 Current article number is invalid - 421 No next article in this group - - Parameters - n Article number - message-id Article message-id - -6.1.4.2. Description - - If the currently selected newsgroup is valid, the current article - number MUST be set to the next article in that newsgroup (that is, - the lowest existing article number greater than the current article - number). If successful, a response indicating the new current - article number and the message-id of that article MUST be returned. - No article text is sent in response to this command. - - If the current article number is already the last article of the - newsgroup, a 421 response MUST be returned. In all other aspects - (apart, of course, from the lack of 422 response), this command is - identical to the LAST command (Section 6.1.3). - -6.1.4.3. Examples - - Example of a successful article retrieval using NEXT: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] NEXT - [S] 223 3000237 <668929@example.org> retrieved - - - - - - - - - - -Feather Standards Track [Page 44] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an attempt to retrieve an article without having selected - a group (via the GROUP command) first: - - [Assumes currently selected newsgroup is invalid.] - [C] NEXT - [S] 412 no newsgroup selected - - Example of an attempt to retrieve an article using the NEXT command - when the current article number is that of the last article in the - group: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] STAT 3002322 - [S] 223 3002322 <411@example.net> retrieved - [C] NEXT - [S] 421 No next article to retrieve - - Example of an attempt to retrieve an article using the NEXT command - when the currently selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] NEXT - [S] 420 No current article selected - -6.2. Retrieval of Articles and Article Sections - - The ARTICLE, BODY, HEAD, and STAT commands are very similar. They - differ only in the parts of the article that are presented to the - client and in the successful response code. The ARTICLE command is - described here in full, while the other three commands are described - in terms of the differences. As specified in Section 3.6, an article - consists of two parts: the article headers and the article body. - - When responding to one of these commands, the server MUST present the - entire article or appropriate part and MUST NOT attempt to alter or - translate it in any way. - - - - - - - - - - - - - -Feather Standards Track [Page 45] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.2.1. ARTICLE - -6.2.1.1. Usage - - Indicating capability: READER - - Syntax - ARTICLE message-id - ARTICLE number - ARTICLE - - Responses - - First form (message-id specified) - 220 0|n message-id Article follows (multi-line) - 430 No article with that message-id - - Second form (article number specified) - 220 n message-id Article follows (multi-line) - 412 No newsgroup selected - 423 No article with that number - - Third form (current article number used) - 220 n message-id Article follows (multi-line) - 412 No newsgroup selected - 420 Current article number is invalid - - Parameters - number Requested article number - n Returned article number - message-id Article message-id - -6.2.1.2. Description - - The ARTICLE command selects an article according to the arguments and - presents the entire article (that is, the headers, an empty line, and - the body, in that order) to the client. The command has three forms. - - In the first form, a message-id is specified, and the server presents - the article with that message-id. In this case, the server MUST NOT - alter the currently selected newsgroup or current article number. - This is both to facilitate the presentation of articles that may be - referenced within another article being read, and because of the - semantic difficulties of determining the proper sequence and - membership of an article that may have been cross-posted to more than - one newsgroup. - - - - - -Feather Standards Track [Page 46] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - In the response, the article number MUST be replaced with zero, - unless there is a currently selected newsgroup and the article is - present in that group, in which case the server MAY use the article's - number in that group. (The server is not required to determine - whether the article is in the currently selected newsgroup or, if so, - what article number it has; the client MUST always be prepared for - zero to be specified.) The server MUST NOT provide an article number - unless use of that number in a second ARTICLE command immediately - following this one would return the same article. Even if the server - chooses to return article numbers in these circumstances, it need not - do so consistently; it MAY return zero to any such command (also see - the STAT examples, Section 6.2.4.3). - - In the second form, an article number is specified. If there is an - article with that number in the currently selected newsgroup, the - server MUST set the current article number to that number. - - In the third form, the article indicated by the current article - number in the currently selected newsgroup is used. - - Note that a previously valid article number MAY become invalid if the - article has been removed. A previously invalid article number MAY - become valid if the article has been reinstated, but this article - number MUST be no less than the reported low water mark for that - group. - - The server MUST NOT change the currently selected newsgroup as a - result of this command. The server MUST NOT change the current - article number except when an article number argument was provided - and the article exists; in particular, it MUST NOT change it - following an unsuccessful response. - - Since the message-id is unique for each article, it may be used by a - client to skip duplicate displays of articles that have been posted - more than once, or to more than one newsgroup. - - The article is returned as a multi-line data block following the 220 - response code. - - If the argument is a message-id and no such article exists, a 430 - response MUST be returned. If the argument is a number or is omitted - and the currently selected newsgroup is invalid, a 412 response MUST - be returned. If the argument is a number and that article does not - exist in the currently selected newsgroup, a 423 response MUST be - returned. If the argument is omitted and the current article number - is invalid, a 420 response MUST be returned. - - - - - -Feather Standards Track [Page 47] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.2.1.3. Examples - - Example of a successful retrieval of an article (explicitly not using - an article number): - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] ARTICLE - [S] 220 3000234 <45223423@example.com> - [S] Path: pathost!demo!whitehouse!not-for-mail - [S] From: "Demo User" - [S] Newsgroups: misc.test - [S] Subject: I am just a test article - [S] Date: 6 Oct 1998 04:38:40 -0500 - [S] Organization: An Example Net, Uncertain, Texas - [S] Message-ID: <45223423@example.com> - [S] - [S] This is just a test article. - [S] . - - Example of a successful retrieval of an article by message-id: - - [C] ARTICLE <45223423@example.com> - [S] 220 0 <45223423@example.com> - [S] Path: pathost!demo!whitehouse!not-for-mail - [S] From: "Demo User" - [S] Newsgroups: misc.test - [S] Subject: I am just a test article - [S] Date: 6 Oct 1998 04:38:40 -0500 - [S] Organization: An Example Net, Uncertain, Texas - [S] Message-ID: <45223423@example.com> - [S] - [S] This is just a test article. - [S] . - - Example of an unsuccessful retrieval of an article by message-id: - - [C] ARTICLE - [S] 430 No Such Article Found - - Example of an unsuccessful retrieval of an article by number: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 news.groups - [C] ARTICLE 300256 - [S] 423 No article with that number - - - - - -Feather Standards Track [Page 48] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an unsuccessful retrieval of an article by number because - no newsgroup was selected first: - - [Assumes currently selected newsgroup is invalid.] - [C] ARTICLE 300256 - [S] 412 No newsgroup selected - - Example of an attempt to retrieve an article when the currently - selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] ARTICLE - [S] 420 No current article selected - -6.2.2. HEAD - -6.2.2.1. Usage - - This command is mandatory. - - Syntax - HEAD message-id - HEAD number - HEAD - - Responses - - First form (message-id specified) - 221 0|n message-id Headers follow (multi-line) - 430 No article with that message-id - - Second form (article number specified) - 221 n message-id Headers follow (multi-line) - 412 No newsgroup selected - 423 No article with that number - - Third form (current article number used) - 221 n message-id Headers follow (multi-line) - 412 No newsgroup selected - 420 Current article number is invalid - - Parameters - number Requested article number - n Returned article number - message-id Article message-id - - - - - -Feather Standards Track [Page 49] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.2.2.2. Description - - The HEAD command behaves identically to the ARTICLE command except - that, if the article exists, the response code is 221 instead of 220 - and only the headers are presented (the empty line separating the - headers and body MUST NOT be included). - -6.2.2.3. Examples - - Example of a successful retrieval of the headers of an article - (explicitly not using an article number): - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HEAD - [S] 221 3000234 <45223423@example.com> - [S] Path: pathost!demo!whitehouse!not-for-mail - [S] From: "Demo User" - [S] Newsgroups: misc.test - [S] Subject: I am just a test article - [S] Date: 6 Oct 1998 04:38:40 -0500 - [S] Organization: An Example Net, Uncertain, Texas - [S] Message-ID: <45223423@example.com> - [S] . - - Example of a successful retrieval of the headers of an article by - message-id: - - [C] HEAD <45223423@example.com> - [S] 221 0 <45223423@example.com> - [S] Path: pathost!demo!whitehouse!not-for-mail - [S] From: "Demo User" - [S] Newsgroups: misc.test - [S] Subject: I am just a test article - [S] Date: 6 Oct 1998 04:38:40 -0500 - [S] Organization: An Example Net, Uncertain, Texas - [S] Message-ID: <45223423@example.com> - [S] . - - Example of an unsuccessful retrieval of the headers of an article by - message-id: - - [C] HEAD - [S] 430 No Such Article Found - - - - - - - -Feather Standards Track [Page 50] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an unsuccessful retrieval of the headers of an article by - number: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HEAD 300256 - [S] 423 No article with that number - - Example of an unsuccessful retrieval of the headers of an article by - number because no newsgroup was selected first: - - [Assumes currently selected newsgroup is invalid.] - [C] HEAD 300256 - [S] 412 No newsgroup selected - - Example of an attempt to retrieve the headers of an article when the - currently selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] HEAD - [S] 420 No current article selected - -6.2.3. BODY - -6.2.3.1. Usage - - Indicating capability: READER - - Syntax - BODY message-id - BODY number - BODY - - Responses - - First form (message-id specified) - 222 0|n message-id Body follows (multi-line) - 430 No article with that message-id - - Second form (article number specified) - 222 n message-id Body follows (multi-line) - 412 No newsgroup selected - 423 No article with that number - - - - - - - -Feather Standards Track [Page 51] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Third form (current article number used) - 222 n message-id Body follows (multi-line) - 412 No newsgroup selected - 420 Current article number is invalid - - Parameters - number Requested article number - n Returned article number - message-id Article message-id - -6.2.3.2. Description - - The BODY command behaves identically to the ARTICLE command except - that, if the article exists, the response code is 222 instead of 220 - and only the body is presented (the empty line separating the headers - and body MUST NOT be included). - -6.2.3.3. Examples - - Example of a successful retrieval of the body of an article - (explicitly not using an article number): - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] BODY - [S] 222 3000234 <45223423@example.com> - [S] This is just a test article. - [S] . - - Example of a successful retrieval of the body of an article by - message-id: - - [C] BODY <45223423@example.com> - [S] 222 0 <45223423@example.com> - [S] This is just a test article. - [S] . - - Example of an unsuccessful retrieval of the body of an article by - message-id: - - [C] BODY - [S] 430 No Such Article Found - - - - - - - - - -Feather Standards Track [Page 52] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an unsuccessful retrieval of the body of an article by - number: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] BODY 300256 - [S] 423 No article with that number - - Example of an unsuccessful retrieval of the body of an article by - number because no newsgroup was selected first: - - [Assumes currently selected newsgroup is invalid.] - [C] BODY 300256 - [S] 412 No newsgroup selected - - Example of an attempt to retrieve the body of an article when the - currently selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] BODY - [S] 420 No current article selected - -6.2.4. STAT - -6.2.4.1. Usage - - This command is mandatory. - - Syntax - STAT message-id - STAT number - STAT - - Responses - - First form (message-id specified) - 223 0|n message-id Article exists - 430 No article with that message-id - - Second form (article number specified) - 223 n message-id Article exists - 412 No newsgroup selected - 423 No article with that number - - - - - - - -Feather Standards Track [Page 53] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Third form (current article number used) - 223 n message-id Article exists - 412 No newsgroup selected - 420 Current article number is invalid - - Parameters - number Requested article number - n Returned article number - message-id Article message-id - -6.2.4.2. Description - - The STAT command behaves identically to the ARTICLE command except - that, if the article exists, it is NOT presented to the client and - the response code is 223 instead of 220. Note that the response is - NOT multi-line. - - This command allows the client to determine whether an article exists - and, in the second and third forms, what its message-id is, without - having to process an arbitrary amount of text. - -6.2.4.3. Examples - - Example of STAT on an existing article (explicitly not using an - article number): - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] STAT - [S] 223 3000234 <45223423@example.com> - - Example of STAT on an existing article by message-id: - - [C] STAT <45223423@example.com> - [S] 223 0 <45223423@example.com> - - Example of STAT on an article not on the server by message-id: - - [C] STAT - [S] 430 No Such Article Found - - Example of STAT on an article not in the server by number: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] STAT 300256 - [S] 423 No article with that number - - - - -Feather Standards Track [Page 54] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of STAT on an article by number when no newsgroup was - selected first: - - [Assumes currently selected newsgroup is invalid.] - [C] STAT 300256 - [S] 412 No newsgroup selected - - Example of STAT on an article when the currently selected newsgroup - is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] STAT - [S] 420 No current article selected - - Example of STAT by message-id on a server that sometimes reports the - actual article number: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] STAT - [S] 223 3000234 <45223423@example.com> - [C] STAT <45223423@example.com> - [S] 223 0 <45223423@example.com> - [C] STAT <45223423@example.com> - [S] 223 3000234 <45223423@example.com> - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] STAT <45223423@example.com> - [S] 223 0 <45223423@example.com> - [C] GROUP alt.crossposts - [S] 211 9999 111111 222222 alt.crossposts - [C] STAT <45223423@example.com> - [S] 223 123456 <45223423@example.com> - [C] STAT - [S] 223 111111 <23894720@example.com> - - The first STAT command establishes the identity of an article in the - group. The second and third show that the server may, but need not, - give the article number when the message-id is specified. The fourth - STAT command shows that zero must be specified if the article isn't - in the currently selected newsgroup. The fifth shows that the - number, if provided, must be that relating to the currently selected - newsgroup. The last one shows that the current article number is - still not changed by the use of STAT with a message-id even if it - returns an article number. - - - - - -Feather Standards Track [Page 55] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.3. Article Posting - - Article posting is done in one of two ways: individual article - posting from news-reading clients using POST, and article transfer - from other news servers using IHAVE. - -6.3.1. POST - -6.3.1.1. Usage - - Indicating capability: POST - - This command MUST NOT be pipelined. - - Syntax - POST - - Responses - - Initial responses - 340 Send article to be posted - 440 Posting not permitted - - Subsequent responses - 240 Article received OK - 441 Posting failed - -6.3.1.2. Description - - If posting is allowed, a 340 response MUST be returned to indicate - that the article to be posted should be sent. If posting is - prohibited for some installation-dependent reason, a 440 response - MUST be returned. - - If posting is permitted, the article MUST be in the format specified - in Section 3.6 and MUST be sent by the client to the server as a - multi-line data block (see Section 3.1.1). Thus a single dot (".") - on a line indicates the end of the text, and lines starting with a - dot in the original text have that dot doubled during transmission. - - Following the presentation of the termination sequence by the client, - the server MUST return a response indicating success or failure of - the article transfer. Note that response codes 340 and 440 are used - in direct response to the POST command while 240 and 441 are returned - after the article is sent. - - - - - - -Feather Standards Track [Page 56] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - A response of 240 SHOULD indicate that, barring unforeseen server - errors, the posted article will be made available on the server - and/or transferred to other servers, as appropriate, possibly - following further processing. In other words, articles not wanted by - the server SHOULD be rejected with a 441 response, rather than being - accepted and then discarded silently. However, the client SHOULD NOT - assume that the article has been successfully transferred unless it - receives an affirmative response from the server and SHOULD NOT - assume that it is being made available to other clients without - explicitly checking (for example, using the STAT command). - - If the session is interrupted before the response is received, it is - possible that an affirmative response was sent but has been lost. - Therefore, in any subsequent session, the client SHOULD either check - whether the article was successfully posted before resending or - ensure that the server will allocate the same message-id to the new - attempt (see Appendix A.2). The latter approach is preferred since - the article might not have been made available for reading yet (for - example, it may have to go through a moderation process). - -6.3.1.3. Examples - - Example of a successful posting: - - [C] POST - [S] 340 Input article; end with . - [C] From: "Demo User" - [C] Newsgroups: misc.test - [C] Subject: I am just a test article - [C] Organization: An Example Net - [C] - [C] This is just a test article. - [C] . - [S] 240 Article received OK - - Example of an unsuccessful posting: - - [C] POST - [S] 340 Input article; end with . - [C] From: "Demo User" - [C] Newsgroups: misc.test - [C] Subject: I am just a test article - [C] Organization: An Example Net - [C] - [C] This is just a test article. - [C] . - [S] 441 Posting failed - - - - -Feather Standards Track [Page 57] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an attempt to post when posting is not allowed: - - [Initial connection set-up completed.] - [S] 201 NNTP Service Ready, posting prohibited - [C] POST - [S] 440 Posting not permitted - -6.3.2. IHAVE - -6.3.2.1. Usage - - Indicating capability: IHAVE - - This command MUST NOT be pipelined. - - Syntax - IHAVE message-id - - Responses - - Initial responses - 335 Send article to be transferred - 435 Article not wanted - 436 Transfer not possible; try again later - - Subsequent responses - 235 Article transferred OK - 436 Transfer failed; try again later - 437 Transfer rejected; do not retry - - Parameters - message-id Article message-id - -6.3.2.2. Description - - The IHAVE command informs the server that the client has an article - with the specified message-id. If the server desires a copy of that - article, a 335 response MUST be returned, instructing the client to - send the entire article. If the server does not want the article - (if, for example, the server already has a copy of it), a 435 - response MUST be returned, indicating that the article is not wanted. - Finally, if the article isn't wanted immediately but the client - should retry later if possible (if, for example, another client is in - the process of sending the same article to the server), a 436 - response MUST be returned. - - - - - - -Feather Standards Track [Page 58] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - If transmission of the article is requested, the client MUST send the - entire article, including headers and body, to the server as a - multi-line data block (see Section 3.1.1). Thus, a single dot (".") - on a line indicates the end of the text, and lines starting with a - dot in the original text have that dot doubled during transmission. - The server MUST return a 235 response, indicating that the article - was successfully transferred; a 436 response, indicating that the - transfer failed but should be tried again later; or a 437 response, - indicating that the article was rejected. - - This function differs from the POST command in that it is intended - for use in transferring already-posted articles between hosts. It - SHOULD NOT be used when the client is a personal news-reading - program, since use of this command indicates that the article has - already been posted at another site and is simply being forwarded - from another host. However, despite this, the server MAY elect not - to post or forward the article if, after further examination of the - article, it deems it inappropriate to do so. Reasons for such - subsequent rejection of an article may include problems such as - inappropriate newsgroups or distributions, disc space limitations, - article lengths, garbled headers, and the like. These are typically - restrictions enforced by the server host's news software and not - necessarily by the NNTP server itself. - - The client SHOULD NOT assume that the article has been successfully - transferred unless it receives an affirmative response from the - server. A lack of response (such as a dropped network connection or - a network timeout) SHOULD be treated the same as a 436 response. - - Because some news server software may not immediately be able to - determine whether an article is suitable for posting or forwarding, - an NNTP server MAY acknowledge the successful transfer of the article - (with a 235 response) but later silently discard it. - - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 59] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -6.3.2.3. Examples - - Example of successfully sending an article to another site: - - [C] IHAVE - [S] 335 Send it; end with . - [C] Path: pathost!demo!somewhere!not-for-mail - [C] From: "Demo User" - [C] Newsgroups: misc.test - [C] Subject: I am just a test article - [C] Date: 6 Oct 1998 04:38:40 -0500 - [C] Organization: An Example Com, San Jose, CA - [C] Message-ID: - [C] - [C] This is just a test article. - [C] . - [S] 235 Article transferred OK - - Example of sending an article to another site that rejects it. Note - that the message-id in the IHAVE command is not the same as the one - in the article headers; while this is bad practice and SHOULD NOT be - done, it is not forbidden. - - [C] IHAVE - [S] 335 Send it; end with . - [C] Path: pathost!demo!somewhere!not-for-mail - [C] From: "Demo User" - [C] Newsgroups: misc.test - [C] Subject: I am just a test article - [C] Date: 6 Oct 1998 04:38:40 -0500 - [C] Organization: An Example Com, San Jose, CA - [C] Message-ID: - [C] - [C] This is just a test article. - [C] . - [S] 437 Article rejected; don't send again - - - - - - - - - - - - - - - -Feather Standards Track [Page 60] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of sending an article to another site where the transfer - fails: - - [C] IHAVE - [S] 335 Send it; end with . - [C] Path: pathost!demo!somewhere!not-for-mail - [C] From: "Demo User" - [C] Newsgroups: misc.test - [C] Subject: I am just a test article - [C] Date: 6 Oct 1998 04:38:40 -0500 - [C] Organization: An Example Com, San Jose, CA - [C] Message-ID: - [C] - [C] This is just a test article. - [C] . - [S] 436 Transfer failed - - Example of sending an article to a site that already has it: - - [C] IHAVE - [S] 435 Duplicate - - Example of sending an article to a site that requests that the - article be tried again later: - - [C] IHAVE - [S] 436 Retry later - -7. Information Commands - - This section lists other commands that may be used at any time - between the beginning of a session and its termination. Using these - commands does not alter any state information, but the response - generated from their use may provide useful information to clients. - -7.1. DATE - -7.1.1. Usage - - Indicating capability: READER - - Syntax - DATE - - Responses - 111 yyyymmddhhmmss Server date and time - - - - - -Feather Standards Track [Page 61] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Parameters - yyyymmddhhmmss Current UTC date and time on server - -7.1.2. Description - - This command exists to help clients find out the current Coordinated - Universal Time [TF.686-1] from the server's perspective. This - command SHOULD NOT be used as a substitute for NTP [RFC1305] but to - provide information that might be useful when using the NEWNEWS - command (see Section 7.4). - - The DATE command MUST return a timestamp from the same clock as is - used for determining article arrival and group creation times (see - Section 6). This clock SHOULD be monotonic, and adjustments SHOULD - be made by running it fast or slow compared to "real" time rather - than by making sudden jumps. A system providing NNTP service SHOULD - keep the system clock as accurate as possible, either with NTP or by - some other method. - - The server MUST return a 111 response specifying the date and time on - the server in the form yyyymmddhhmmss. This date and time is in - Coordinated Universal Time. - -7.1.3. Examples - - [C] DATE - [S] 111 19990623135624 - -7.2. HELP - -7.2.1. Usage - - This command is mandatory. - - Syntax - HELP - - Responses - 100 Help text follows (multi-line) - -7.2.2. Description - - This command provides a short summary of the commands that are - understood by this implementation of the server. The help text will - be presented as a multi-line data block following the 100 response - code. - - - - - -Feather Standards Track [Page 62] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - This text is not guaranteed to be in any particular format (but must - be UTF-8) and MUST NOT be used by clients as a replacement for the - CAPABILITIES command described in Section 5.2. - -7.2.3. Examples - - [C] HELP - [S] 100 Help text follows - [S] This is some help text. There is no specific - [S] formatting requirement for this test, though - [S] it is customary for it to list the valid commands - [S] and give a brief definition of what they do. - [S] . - -7.3. NEWGROUPS - -7.3.1. Usage - - Indicating capability: READER - - Syntax - NEWGROUPS date time [GMT] - - Responses - 231 List of new newsgroups follows (multi-line) - - Parameters - date Date in yymmdd or yyyymmdd format - time Time in hhmmss format - -7.3.2. Description - - This command returns a list of newsgroups created on the server since - the specified date and time. The results are in the same format as - the LIST ACTIVE command (see Section 7.6.3). However, they MAY - include groups not available on the server (and so not returned by - LIST ACTIVE) and MAY omit groups for which the creation date is not - available. - - The date is specified as 6 or 8 digits in the format [xx]yymmdd, - where xx is the first two digits of the year (19-99), yy is the last - two digits of the year (00-99), mm is the month (01-12), and dd is - the day of the month (01-31). Clients SHOULD specify all four digits - of the year. If the first two digits of the year are not specified - (this is supported only for backward compatibility), the year is to - be taken from the current century if yy is smaller than or equal to - the current year, and the previous century otherwise. - - - - -Feather Standards Track [Page 63] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The time is specified as 6 digits in the format hhmmss, where hh is - the hours in the 24-hour clock (00-23), mm is the minutes (00-59), - and ss is the seconds (00-60, to allow for leap seconds). The token - "GMT" specifies that the date and time are given in Coordinated - Universal Time [TF.686-1]; if it is omitted, then the date and time - are specified in the server's local timezone. Note that there is no - way of using the protocol specified in this document to establish the - server's local timezone. - - Note that an empty list is a possible valid response and indicates - that there are no new newsgroups since that date-time. - - Clients SHOULD make all queries using Coordinated Universal Time - (i.e., by including the "GMT" argument) when possible. - -7.3.3. Examples - - Example where there are new groups: - - [C] NEWGROUPS 19990624 000000 GMT - [S] 231 list of new newsgroups follows - [S] alt.rfc-writers.recovery 4 1 y - [S] tx.natives.recovery 89 56 y - [S] . - - Example where there are no new groups: - - [C] NEWGROUPS 19990624 000000 GMT - [S] 231 list of new newsgroups follows - [S] . - -7.4. NEWNEWS - -7.4.1. Usage - - Indicating capability: NEWNEWS - - Syntax - NEWNEWS wildmat date time [GMT] - - Responses - 230 List of new articles follows (multi-line) - - Parameters - wildmat Newsgroups of interest - date Date in yymmdd or yyyymmdd format - time Time in hhmmss format - - - - -Feather Standards Track [Page 64] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -7.4.2. Description - - This command returns a list of message-ids of articles posted or - received on the server, in the newsgroups whose names match the - wildmat, since the specified date and time. One message-id is sent - on each line; the order of the response has no specific significance - and may vary from response to response in the same session. A - message-id MAY appear more than once; if it does, it has the same - meaning as if it appeared only once. - - Date and time are in the same format as the NEWGROUPS command (see - Section 7.3). - - Note that an empty list is a possible valid response and indicates - that there is currently no new news in the relevant groups. - - Clients SHOULD make all queries in Coordinated Universal Time (i.e., - by using the "GMT" argument) when possible. - -7.4.3. Examples - - Example where there are new articles: - - [C] NEWNEWS news.*,sci.* 19990624 000000 GMT - [S] 230 list of new articles by message-id follows - [S] - [S] - [S] . - - Example where there are no new articles: - - [C] NEWNEWS alt.* 19990624 000000 GMT - [S] 230 list of new articles by message-id follows - [S] . - -7.5. Time - - As described in Section 6, each article has an arrival timestamp. - Each newsgroup also has a creation timestamp. These timestamps are - used by the NEWNEWS and NEWGROUP commands to construct their - responses. - - Clients can ensure that they do not have gaps in lists of articles or - groups by using the DATE command in the following manner: - - First session: - Issue DATE command and record result. - Issue NEWNEWS command using a previously chosen timestamp. - - - -Feather Standards Track [Page 65] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Subsequent sessions: - Issue DATE command and hold result in temporary storage. - Issue NEWNEWS command using timestamp saved from previous session. - Overwrite saved timestamp with that currently in temporary - storage. - - In order to allow for minor errors, clients MAY want to adjust the - timestamp back by two or three minutes before using it in NEWNEWS. - -7.5.1. Examples - - First session: - - [C] DATE - [S] 111 20010203112233 - [C] NEWNEWS local.chat 20001231 235959 GMT - [S] 230 list follows - [S] - [S] - [S] - [S] . - - Second session (the client has subtracted 3 minutes from the - timestamp returned previously): - - [C] DATE - [S] 111 20010204003344 - [C] NEWNEWS local.chat 20010203 111933 GMT - [S] 230 list follows - [S] - [S] - [S] - [S] . - - Note how arrived in the 3 minute gap and so - is listed in both responses. - -7.6. The LIST Commands - - The LIST family of commands all return information that is multi-line - and that can, in general, be expected not to change during the - session. Often the information is related to newsgroups, in which - case the response has one line per newsgroup and a wildmat MAY be - provided to restrict the groups for which information is returned. - - The set of available keywords (including those provided by - extensions) is given in the capability list with capability label - LIST. - - - -Feather Standards Track [Page 66] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -7.6.1. LIST - -7.6.1.1. Usage - - Indicating capability: LIST - - Syntax - LIST [keyword [wildmat|argument]] - - Responses - 215 Information follows (multi-line) - - Parameters - keyword Information requested [1] - argument Specific to keyword - wildmat Groups of interest - - [1] If no keyword is provided, it defaults to ACTIVE. - -7.6.1.2. Description - - The LIST command allows the server to provide blocks of information - to the client. This information may be global or may be related to - newsgroups; in the latter case, the information may be returned - either for all groups or only for those matching a wildmat. Each - block of information is represented by a different keyword. The - command returns the specific information identified by the keyword. - - If the information is available, it is returned as a multi-line data - block following the 215 response code. The format of the information - depends on the keyword. The information MAY be affected by the - additional argument, but the format MUST NOT be. - - If the information is based on newsgroups and the optional wildmat - argument is specified, the response is limited to only the groups (if - any) whose names match the wildmat and for which the information is - available. - - Note that an empty list is a possible valid response; for a - newsgroup-based keyword, it indicates that there are no groups - meeting the above criteria. - - If the keyword is not recognised, or if an argument is specified and - the keyword does not expect one, a 501 response code MUST BE - returned. If the keyword is recognised but the server does not - maintain the information, a 503 response code MUST BE returned. - - - - - -Feather Standards Track [Page 67] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The LIST command MUST NOT change the visible state of the server in - any way; that is, the behaviour of subsequent commands MUST NOT be - affected by whether the LIST command was issued. For example, it - MUST NOT make groups available that otherwise would not have been. - -7.6.1.3. Examples - - Example of LIST with the ACTIVE keyword: - - [C] LIST ACTIVE - [S] 215 list of newsgroups follows - [S] misc.test 3002322 3000234 y - [S] comp.risks 442001 441099 m - [S] alt.rfc-writers.recovery 4 1 y - [S] tx.natives.recovery 89 56 y - [S] tx.natives.recovery.d 11 9 n - [S] . - - Example of LIST with no keyword: - - [C] LIST - [S] 215 list of newsgroups follows - [S] misc.test 3002322 3000234 y - [S] comp.risks 442001 441099 m - [S] alt.rfc-writers.recovery 4 1 y - [S] tx.natives.recovery 89 56 y - [S] tx.natives.recovery.d 11 9 n - [S] . - - The output is identical to that of the previous example. - - Example of LIST on a newsgroup-based keyword with and without - wildmat: - - [C] LIST ACTIVE.TIMES - [S] 215 information follows - [S] misc.test 930445408 - [S] alt.rfc-writers.recovery 930562309 - [S] tx.natives.recovery 930678923 - [S] . - [C] LIST ACTIVE.TIMES tx.* - [S] 215 information follows - [S] tx.natives.recovery 930678923 - [S] . - - - - - - - -Feather Standards Track [Page 68] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of LIST returning an error where the keyword is recognized - but the software does not maintain this information: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] LIST ACTIVE NEWSGROUPS ACTIVE.TIMES XTRA.DATA - [S] . - [C] LIST XTRA.DATA - [S] 503 Data item not stored - - Example of LIST where the keyword is not recognised: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] LIST ACTIVE NEWSGROUPS ACTIVE.TIMES XTRA.DATA - [S] . - [C] LIST DISTRIB.PATS - [S] 501 Syntax Error - -7.6.2. Standard LIST Keywords - - This specification defines the following LIST keywords: - - +--------------+---------------+------------------------------------+ - | Keyword | Definition | Status | - +--------------+---------------+------------------------------------+ - | ACTIVE | Section 7.6.3 | Mandatory if the READER capability | - | | | is advertised | - | | | | - | ACTIVE.TIMES | Section 7.6.4 | Optional | - | | | | - | DISTRIB.PATS | Section 7.6.5 | Optional | - | | | | - | HEADERS | Section 8.6 | Mandatory if the HDR capability is | - | | | advertised | - | | | | - | NEWSGROUPS | Section 7.6.6 | Mandatory if the READER capability | - | | | is advertised | - | | | | - | OVERVIEW.FMT | Section 8.4 | Mandatory if the OVER capability | - | | | is advertised | - +--------------+---------------+------------------------------------+ - - - - - -Feather Standards Track [Page 69] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Where one of these LIST keywords is supported by a server, it MUST - have the meaning given in the relevant sub-section. - -7.6.3. LIST ACTIVE - - This keyword MUST be supported by servers advertising the READER - capability. - - LIST ACTIVE returns a list of valid newsgroups and associated - information. If no wildmat is specified, the server MUST include - every group that the client is permitted to select with the GROUP - command (Section 6.1.1). Each line of this list consists of four - fields separated from each other by one or more spaces: - - o The name of the newsgroup. - o The reported high water mark for the group. - o The reported low water mark for the group. - o The current status of the group on this server. - - The reported high and low water marks are as described in the GROUP - command (see Section 6.1.1), but note that they are in the opposite - order to the 211 response to that command. - - The status field is typically one of the following: - - "y" Posting is permitted. - - "n" Posting is not permitted. - - "m" Postings will be forwarded to the newsgroup moderator. - - The server SHOULD use these values when these meanings are required - and MUST NOT use them with any other meaning. Other values for the - status may exist; the definition of these other values and the - circumstances under which they are returned may be specified in an - extension or may be private to the server. A client SHOULD treat an - unrecognized status as giving no information. - - The status of a newsgroup only indicates how posts to that newsgroup - are normally processed and is not necessarily customised to the - specific client. For example, if the current client is forbidden - from posting, then this will apply equally to groups with status "y". - Conversely, a client with special privileges (not defined by this - specification) might be able to post to a group with status "n". - - - - - - - -Feather Standards Track [Page 70] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - For example: - - [C] LIST ACTIVE - [S] 215 list of newsgroups follows - [S] misc.test 3002322 3000234 y - [S] comp.risks 442001 441099 m - [S] alt.rfc-writers.recovery 4 1 y - [S] tx.natives.recovery 89 56 y - [S] tx.natives.recovery.d 11 9 n - [S] . - - or, on an implementation that includes leading zeroes: - - [C] LIST ACTIVE - [S] 215 list of newsgroups follows - [S] misc.test 0003002322 0003000234 y - [S] comp.risks 0000442001 0000441099 m - [S] alt.rfc-writers.recovery 0000000004 0000000001 y - [S] tx.natives.recovery 0000000089 0000000056 y - [S] tx.natives.recovery.d 0000000011 0000000009 n - [S] . - - The information is newsgroup based, and a wildmat MAY be specified, - in which case the response is limited to only the groups (if any) - whose names match the wildmat. For example: - - [C] LIST ACTIVE *.recovery - [S] 215 list of newsgroups follows - [S] alt.rfc-writers.recovery 4 1 y - [S] tx.natives.recovery 89 56 y - [S] . - -7.6.4. LIST ACTIVE.TIMES - - This keyword is optional. - - The active.times list is maintained by some NNTP servers to contain - information about who created a particular newsgroup and when. Each - line of this list consists of three fields separated from each other - by one or more spaces. The first field is the name of the newsgroup. - The second is the time when this group was created on this news - server, measured in seconds since the start of January 1, 1970. The - third is plain text intended to describe the entity that created the - newsgroup; it is often a mailbox as defined in RFC 2822 [RFC2822]. - For example: - - - - - - -Feather Standards Track [Page 71] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - [C] LIST ACTIVE.TIMES - [S] 215 information follows - [S] misc.test 930445408 - [S] alt.rfc-writers.recovery 930562309 - [S] tx.natives.recovery 930678923 - [S] . - - The list MAY omit newsgroups for which the information is unavailable - and MAY include groups not available on the server; in particular, it - MAY omit all groups created before the date and time of the oldest - entry. The client MUST NOT assume that the list is complete or that - it matches the list returned by the LIST ACTIVE command - (Section 7.6.3). The NEWGROUPS command (Section 7.3) may provide a - better way to access this information, and the results of the two - commands SHOULD be consistent except that, if the latter is invoked - with a date and time earlier than the oldest entry in active.times - list, its result may include extra groups. - - The information is newsgroup based, and a wildmat MAY be specified, - in which case the response is limited to only the groups (if any) - whose names match the wildmat. - -7.6.5. LIST DISTRIB.PATS - - This keyword is optional. - - The distrib.pats list is maintained by some NNTP servers to assist - clients to choose a value for the content of the Distribution header - of a news article being posted. Each line of this list consists of - three fields separated from each other by a colon (":"). The first - field is a weight, the second field is a wildmat (which may be a - simple newsgroup name), and the third field is a value for the - Distribution header content. For example: - - [C] LIST DISTRIB.PATS - [S] 215 information follows - [S] 10:local.*:local - [S] 5:*:world - [S] 20:local.here.*:thissite - [S] . - - The client MAY use this information to construct an appropriate - Distribution header given the name of a newsgroup. To do so, it - should determine the lines whose second field matches the newsgroup - name, select from among them the line with the highest weight (with 0 - being the lowest), and use the value of the third field to construct - the Distribution header. - - - - -Feather Standards Track [Page 72] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - The information is not newsgroup based, and an argument MUST NOT be - specified. - -7.6.6. LIST NEWSGROUPS - - This keyword MUST be supported by servers advertising the READER - capability. - - The newsgroups list is maintained by NNTP servers to contain the name - of each newsgroup that is available on the server and a short - description about the purpose of the group. Each line of this list - consists of two fields separated from each other by one or more space - or TAB characters (the usual practice is a single TAB). The first - field is the name of the newsgroup, and the second is a short - description of the group. For example: - - [C] LIST NEWSGROUPS - [S] 215 information follows - [S] misc.test General Usenet testing - [S] alt.rfc-writers.recovery RFC Writers Recovery - [S] tx.natives.recovery Texas Natives Recovery - [S] . - - The list MAY omit newsgroups for which the information is unavailable - and MAY include groups not available on the server. The client MUST - NOT assume that the list is complete or that it matches the list - returned by LIST ACTIVE. - - The description SHOULD be in UTF-8. However, servers often obtain - the information from external sources. These sources may have used - different encodings (ones that use octets in the range 128 to 255 in - some other manner) and, in that case, the server MAY pass it on - unchanged. Therefore, clients MUST be prepared to receive such - descriptions. - - The information is newsgroup based, and a wildmat MAY be specified, - in which case the response is limited to only the groups (if any) - whose names match the wildmat. - -8. Article Field Access Commands - - This section lists commands that may be used to access specific - article fields; that is, headers of articles and metadata about - articles. These commands typically fetch data from an "overview - database", which is a database of headers extracted from incoming - articles plus metadata determined as the article arrives. Only - certain fields are included in the database. - - - - -Feather Standards Track [Page 73] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - This section is based on the Overview/NOV database [ROBE1995] - developed by Geoff Collyer. - -8.1. Article Metadata - - Article "metadata" is data about articles that does not occur within - the article itself. Each metadata item has a name that MUST begin - with a colon (and that MUST NOT contain a colon elsewhere within it). - As with header names, metadata item names are not case sensitive. - - When generating a metadata item, the server MUST compute it for - itself and MUST NOT trust any related value provided in the article. - (In particular, a Lines or Bytes header in the article MUST NOT be - assumed to specify the correct number of lines or bytes in the - article.) If the server has access to several non-identical copies - of an article, the value returned MUST be correct for any copy of - that article retrieved during the same session. - - This specification defines two metadata items: ":bytes" and ":lines". - Other metadata items may be defined by extensions. The names of - metadata items defined by registered extensions MUST NOT begin with - ":x-". To avoid the risk of a clash with a future registered - extension, the names of metadata items defined by private extensions - SHOULD begin with ":x-". - -8.1.1. The :bytes Metadata Item - - The :bytes metadata item for an article is a decimal integer. It - SHOULD equal the number of octets in the entire article: headers, - body, and separating empty line (counting a CRLF pair as two octets, - and excluding both the "." CRLF terminating the response and any "." - added for "dot-stuffing" purposes). - - Note to client implementers: some existing servers return a value - different from that above. The commonest reasons for this are as - follows: - - o Counting a CRLF pair as one octet. - - o Including the "." character used for dot-stuffing in the number. - - o Including the terminating "." CRLF in the number. - - o Using one copy of an article for counting the octets but then - returning another one that differs in some (permitted) manner. - - Implementations should be prepared for such variation and MUST NOT - rely on the value being accurate. - - - -Feather Standards Track [Page 74] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -8.1.2. The :lines Metadata Item - - The :lines metadata item for an article is a decimal integer. It - MUST equal the number of lines in the article body (excluding the - empty line separating headers and body). Equivalently, it is two - less than the number of CRLF pairs that the BODY command would return - for that article (the extra two are those following the response code - and the termination octet). - -8.2. Database Consistency - - The information stored in the overview database may change over time. - If the database records the content or absence of a given field (that - is, a header or metadata item) for all articles, it is said to be - "consistent" for that field. If it records the content of a header - for some articles but not for others that nevertheless included that - header, or if it records a metadata item for some articles but not - for others to which that item applies, it is said to be - "inconsistent" for that field. - - The LIST OVERVIEW.FMT command SHOULD list all the fields for which - the database is consistent at that moment. It MAY omit such fields - (for example, if it is not known whether the database is consistent - or inconsistent). It MUST NOT include fields for which the database - is inconsistent or that are not stored in the database. Therefore, - if a header appears in the LIST OVERVIEW.FMT output but not in the - OVER output for a given article, that header does not appear in the - article (similarly for metadata items). - - These rules assume that the fields being stored in the database - remain constant for long periods of time, and therefore the database - will be consistent. When the set of fields to be stored is changed, - it will be inconsistent until either the database is rebuilt or the - only articles remaining are those received since the change. - Therefore, the output from LIST OVERVIEW.FMT needs to be altered - twice. Firstly, before any fields stop being stored they MUST be - removed from the output; then, when the database is once more known - to be consistent, the new fields SHOULD be added to the output. - - If the HDR command uses the overview database rather than taking - information directly from the articles, the same issues of - consistency and inconsistency apply, and the LIST HEADERS command - SHOULD take the same approach as the LIST OVERVIEW.FMT command in - resolving them. - - - - - - - -Feather Standards Track [Page 75] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -8.3. OVER - -8.3.1. Usage - - Indicating capability: OVER - - Syntax - OVER message-id - OVER range - OVER - - Responses - - First form (message-id specified) - 224 Overview information follows (multi-line) - 430 No article with that message-id - - Second form (range specified) - 224 Overview information follows (multi-line) - 412 No newsgroup selected - 423 No articles in that range - - Third form (current article number used) - 224 Overview information follows (multi-line) - 412 No newsgroup selected - 420 Current article number is invalid - - Parameters - range Number(s) of articles - message-id Message-id of article - -8.3.2. Description - - The OVER command returns the contents of all the fields in the - database for an article specified by message-id, or from a specified - article or range of articles in the currently selected newsgroup. - - The message-id argument indicates a specific article. The range - argument may be any of the following: - - o An article number. - - o An article number followed by a dash to indicate all following. - - o An article number followed by a dash followed by another article - number. - - If neither is specified, the current article number is used. - - - -Feather Standards Track [Page 76] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Support for the first (message-id) form is optional. If it is - supported, the OVER capability line MUST include the argument - "MSGID". Otherwise, the capability line MUST NOT include this - argument, and the OVER command MUST return the generic response code - 503 when this form is used. - - If the information is available, it is returned as a multi-line data - block following the 224 response code and contains one line per - article, sorted in numerical order of article number. (Note that - unless the argument is a range including a dash, there will be - exactly one line in the data block.) Each line consists of a number - of fields separated by a TAB. A field may be empty (in which case - there will be two adjacent TABs), and a sequence of trailing TABs may - be omitted. - - The first 8 fields MUST be the following, in order: - - "0" or article number (see below) - Subject header content - From header content - Date header content - Message-ID header content - References header content - :bytes metadata item - :lines metadata item - - If the article is specified by message-id (the first form of the - command), the article number MUST be replaced with zero, except that - if there is a currently selected newsgroup and the article is present - in that group, the server MAY use the article's number in that group. - (See the ARTICLE command (Section 6.2.1) and STAT examples - (Section 6.2.4.3) for more details.) In the other two forms of the - command, the article number MUST be returned. - - Any subsequent fields are the contents of the other headers and - metadata held in the database. - - For the five mandatory headers, the content of each field MUST be - based on the content of the header (that is, with the header name and - following colon and space removed). If the article does not contain - that header, or if the content is empty, the field MUST be empty. - For the two mandatory metadata items, the content of the field MUST - be just the value, with no other text. - - For all subsequent fields that contain headers, the content MUST be - the entire header line other than the trailing CRLF. For all - subsequent fields that contain metadata, the field consists of the - metadata name, a single space, and then the value. - - - -Feather Standards Track [Page 77] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - For all fields, the value is processed by first removing all CRLF - pairs (that is, undoing any folding and removing the terminating - CRLF) and then replacing each TAB with a single space. If there is - no such header in the article, no such metadata item, or no header or - item stored in the database for that article, the corresponding field - MUST be empty. - - Note that, after unfolding, the characters NUL, LF, and CR cannot - occur in the header of an article offered by a conformant server. - Nevertheless, servers SHOULD check for these characters and replace - each one by a single space (so that, for example, CR LF LF TAB will - become two spaces, since the CR and first LF will be removed by the - unfolding process). This will encourage robustness in the face of - non-conforming data; it is also possible that future versions of this - specification could permit these characters to appear in articles. - - The server SHOULD NOT produce output for articles that no longer - exist. - - If the argument is a message-id and no such article exists, a 430 - response MUST be returned. If the argument is a range or is omitted - and the currently selected newsgroup is invalid, a 412 response MUST - be returned. If the argument is a range and no articles in that - number range exist in the currently selected newsgroup, including the - case where the second number is less than the first one, a 423 - response MUST be returned. If the argument is omitted and the - current article number is invalid, a 420 response MUST be returned. - -8.3.3. Examples - - In the first four examples, TAB has been replaced by vertical bar and - some lines have been folded for readability. - - Example of a successful retrieval of overview information for an - article (explicitly not using an article number): - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] OVER - [S] 224 Overview information follows - [S] 3000234|I am just a test article|"Demo User" - |6 Oct 1998 04:38:40 -0500| - <45223423@example.com>|<45454@example.net>|1234| - 17|Xref: news.example.com misc.test:3000363 - [S] . - - - - - - -Feather Standards Track [Page 78] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of a successful retrieval of overview information for an - article by message-id: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] OVER MSGID - [S] LIST ACTIVE NEWSGROUPS OVERVIEW.FMT - [S] . - [C] OVER <45223423@example.com> - [S] 224 Overview information follows - [S] 0|I am just a test article|"Demo User" - |6 Oct 1998 04:38:40 -0500| - <45223423@example.com>|<45454@example.net>|1234| - 17|Xref: news.example.com misc.test:3000363 - [S] . - - Note that the article number has been replaced by "0". - - Example of the same commands on a system that does not implement - retrieval by message-id: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] OVER - [S] LIST ACTIVE NEWSGROUPS OVERVIEW.FMT - [S] . - [C] OVER <45223423@example.com> - [S] 503 Overview by message-id unsupported - - - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 79] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of a successful retrieval of overview information for a range - of articles: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] OVER 3000234-3000240 - [S] 224 Overview information follows - [S] 3000234|I am just a test article|"Demo User" - |6 Oct 1998 04:38:40 -0500| - <45223423@example.com>|<45454@example.net>|1234| - 17|Xref: news.example.com misc.test:3000363 - [S] 3000235|Another test article|nobody@nowhere.to - (Demo User)|6 Oct 1998 04:38:45 -0500|<45223425@to.to>|| - 4818|37||Distribution: fi - [S] 3000238|Re: I am just a test article|somebody@elsewhere.to| - 7 Oct 1998 11:38:40 +1200|| - <45223423@to.to>|9234|51 - [S] . - - Note the missing "References" and Xref headers in the second line, - the missing trailing fields in the first and last lines, and that - there are only results for those articles that still exist. - - Example of an unsuccessful retrieval of overview information on an - article by number: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] OVER 300256 - [S] 423 No such article in this group - - Example of an invalid range: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] OVER 3000444-3000222 - [S] 423 Empty range - - Example of an unsuccessful retrieval of overview information by - number because no newsgroup was selected first: - - [Assumes currently selected newsgroup is invalid.] - [C] OVER - [S] 412 No newsgroup selected - - - - - - - -Feather Standards Track [Page 80] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an attempt to retrieve information when the currently - selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] OVER - [S] 420 No current article selected - -8.4. LIST OVERVIEW.FMT - -8.4.1. Usage - - Indicating capability: OVER - - Syntax - LIST OVERVIEW.FMT - - Responses - 215 Information follows (multi-line) - -8.4.2. Description - - See Section 7.6.1 for general requirements of the LIST command. - - The LIST OVERVIEW.FMT command returns a description of the fields in - the database for which it is consistent (as described above). The - information is returned as a multi-line data block following the 215 - response code. The information contains one line per field in the - order in which they are returned by the OVER command; the first 7 - lines MUST (except for the case of letters) be exactly as follows: - - Subject: - From: - Date: - Message-ID: - References: - :bytes - :lines - - For compatibility with existing implementations, the last two lines - MAY instead be: - - Bytes: - Lines: - - even though they refer to metadata, not headers. - - - - - -Feather Standards Track [Page 81] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - All subsequent lines MUST consist of either a header name followed by - ":full", or the name of a piece of metadata. - - There are no leading or trailing spaces in the output. - - Note that the 7 fixed lines describe the 2nd to 8th fields of the - OVER output. The "full" suffix (which may use either uppercase, - lowercase, or a mix) is a reminder that the corresponding fields - include the header name. - - This command MAY generate different results if it is used more than - once in a session. - - If the OVER command is not implemented, the meaning of the output - from this command is not specified, but it must still meet the above - syntactic requirements. - -8.4.3. Examples - - Example of LIST OVERVIEW.FMT output corresponding to the example OVER - output above, in the preferred format: - - [C] LIST OVERVIEW.FMT - [S] 215 Order of fields in overview database. - [S] Subject: - [S] From: - [S] Date: - [S] Message-ID: - [S] References: - [S] :bytes - [S] :lines - [S] Xref:full - [S] Distribution:full - [S] . - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 82] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of LIST OVERVIEW.FMT output corresponding to the example OVER - output above, in the alternative format: - - [C] LIST OVERVIEW.FMT - [S] 215 Order of fields in overview database. - [S] Subject: - [S] From: - [S] Date: - [S] Message-ID: - [S] References: - [S] Bytes: - [S] Lines: - [S] Xref:FULL - [S] Distribution:FULL - [S] . - -8.5. HDR - -8.5.1. Usage - - Indicating capability: HDR - - Syntax - HDR field message-id - HDR field range - HDR field - - Responses - - First form (message-id specified) - 225 Headers follow (multi-line) - 430 No article with that message-id - - Second form (range specified) - 225 Headers follow (multi-line) - 412 No newsgroup selected - 423 No articles in that range - - Third form (current article number used) - 225 Headers follow (multi-line) - 412 No newsgroup selected - 420 Current article number is invalid - - Parameters - field Name of field - range Number(s) of articles - message-id Message-id of article - - - - -Feather Standards Track [Page 83] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -8.5.2. Description - - The HDR command provides access to specific fields from an article - specified by message-id, or from a specified article or range of - articles in the currently selected newsgroup. It MAY take the - information directly from the articles or from the overview database. - In the case of headers, an implementation MAY restrict the use of - this command to a specific list of headers or MAY allow it to be used - with any header; it may behave differently when it is used with a - message-id argument and when it is used with a range or no argument. - - The required field argument is the name of a header with the colon - omitted (e.g., "subject") or the name of a metadata item including - the leading colon (e.g., ":bytes"), and is case insensitive. - - The message-id argument indicates a specific article. The range - argument may be any of the following: - - o An article number. - - o An article number followed by a dash to indicate all following. - - o An article number followed by a dash followed by another article - number. - - If neither is specified, the current article number is used. - - If the information is available, it is returned as a multi-line data - block following the 225 response code and contains one line for each - article in the range that exists. (Note that unless the argument is - a range including a dash, there will be exactly one line in the data - block.) The line consists of the article number, a space, and then - the contents of the field. In the case of a header, the header name, - the colon, and the first space after the colon are all omitted. - - If the article is specified by message-id (the first form of the - command), the article number MUST be replaced with zero, except that - if there is a currently selected newsgroup and the article is present - in that group, the server MAY use the article's number in that group. - (See the ARTICLE command (Section 6.2.1) and STAT examples - (Section 6.2.4.3) for more details.) In the other two forms of the - command, the article number MUST be returned. - - Header contents are modified as follows: all CRLF pairs are removed, - and then each TAB is replaced with a single space. (Note that this - is the same transformation as is performed by the OVER command - (Section 8.3.2), and the same comment concerning NUL, CR, and LF - applies.) - - - -Feather Standards Track [Page 84] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Note the distinction between headers and metadata appearing to have - the same meaning. Headers are always taken unchanged from the - article; metadata are always calculated. For example, a request for - "Lines" returns the contents of the "Lines" header of the specified - articles, if any, no matter whether they accurately state the number - of lines, while a request for ":lines" returns the line count - metadata, which is always the actual number of lines irrespective of - what any header may state. - - If the requested header is not present in the article, or if it is - present but empty, a line for that article is included in the output, - but the header content portion of the line is empty (the space after - the article number MAY be retained or omitted). If the header occurs - in a given article more than once, only the content of the first - occurrence is returned by HDR. If any article number in the provided - range does not exist in the group, no line for that article number is - included in the output. - - If the second argument is a message-id and no such article exists, a - 430 response MUST be returned. If the second argument is a range or - is omitted and the currently selected newsgroup is invalid, a 412 - response MUST be returned. If the second argument is a range and no - articles in that number range exist in the currently selected - newsgroup, including the case where the second number is less than - the first one, a 423 response MUST be returned. If the second - argument is omitted and the current article number is invalid, a 420 - response MUST be returned. - - A server MAY only allow HDR commands for a limited set of fields; it - may behave differently in this respect for the first (message-id) - form from how it would for the other forms. If so, it MUST respond - with the generic 503 response to attempts to request other fields, - rather than return erroneous results, such as a successful empty - response. - - If HDR uses the overview database and it is inconsistent for the - requested field, the server MAY return what results it can, or it MAY - respond with the generic 503 response. In the latter case, the field - MUST NOT appear in the output from LIST HEADERS. - - - - - - - - - - - - -Feather Standards Track [Page 85] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -8.5.3. Examples - - Example of a successful retrieval of subject lines from a range of - articles (3000235 has no Subject header, and 3000236 is missing): - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HDR Subject 3000234-3000238 - [S] 225 Headers follow - [S] 3000234 I am just a test article - [S] 3000235 - [S] 3000237 Re: I am just a test article - [S] 3000238 Ditto - [S] . - - Example of a successful retrieval of line counts from a range of - articles: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HDR :lines 3000234-3000238 - [S] 225 Headers follow - [S] 3000234 42 - [S] 3000235 5 - [S] 3000237 11 - [S] 3000238 2378 - [S] . - - Example of a successful retrieval of the subject line from an article - by message-id: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HDR subject - [S] 225 Header information follows - [S] 0 I am just a test article - [S] . - - Example of a successful retrieval of the subject line from the - current article: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HDR subject - [S] 225 Header information follows - [S] 3000234 I am just a test article - [S] . - - - - -Feather Standards Track [Page 86] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an unsuccessful retrieval of a header from an article by - message-id: - - [C] HDR subject - [S] 430 No Such Article Found - - Example of an unsuccessful retrieval of headers from articles by - number because no newsgroup was selected first: - - [Assumes currently selected newsgroup is invalid.] - [C] HDR subject 300256- - [S] 412 No newsgroup selected - - Example of an unsuccessful retrieval of headers because the currently - selected newsgroup is empty: - - [C] GROUP example.empty.newsgroup - [S] 211 0 0 0 example.empty.newsgroup - [C] HDR subject 1- - [S] 423 No articles in that range - - Example of an unsuccessful retrieval of headers because the server - does not allow HDR commands for that header: - - [C] GROUP misc.test - [S] 211 1234 3000234 3002322 misc.test - [C] HDR Content-Type 3000234-3000238 - [S] 503 HDR not permitted on Content-Type - -8.6. LIST HEADERS - -8.6.1. Usage - - Indicating capability: HDR - - Syntax - LIST HEADERS [MSGID|RANGE] - - Responses - 215 Field list follows (multi-line) - - Parameters - MSGID Requests list for access by message-id - RANGE Requests list for access by range - - - - - - - -Feather Standards Track [Page 87] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -8.6.2. Description - - See Section 7.6.1 for general requirements of the LIST command. - - The LIST HEADERS command returns a list of fields that may be - retrieved using the HDR command. - - The information is returned as a multi-line data block following the - 215 response code and contains one line for each field name - (excluding the trailing colon for headers and including the leading - colon for metadata items). If the implementation allows any header - to be retrieved, it MUST NOT include any header names in the list but - MUST include the special entry ":" (a single colon on its own). It - MUST still explicitly list any metadata items that are available. - The order of items in the list is not significant; the server need - not even consistently return the same order. The list MAY be empty - (though in this circumstance there is little point in providing the - HDR command). - - An implementation that also supports the OVER command SHOULD at least - permit all the headers and metadata items listed in the output from - the LIST OVERVIEW.FMT command. - - If the server treats the first form of the HDR command (message-id - specified) differently from the other two forms (range specified or - current article number used) in respect of which headers or metadata - items are available, then the following apply: - - o If the MSGID argument is specified, the results MUST be those - available for the first form of the HDR command. - - o If the RANGE argument is specified, the results MUST be those - available for the second and third forms of the HDR command. - - o If no argument is specified, the results MUST be those available - in all forms of the HDR command (that is, it MUST only list those - items listed in both the previous cases). - - If the server does not treat the various forms differently, then it - MUST ignore any argument and always produce the same results (though - not necessarily always in the same order). - - If the HDR command is not implemented, the meaning of the output from - this command is not specified, but it must still meet the above - syntactic requirements. - - - - - - -Feather Standards Track [Page 88] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -8.6.3. Examples - - Example of an implementation providing access to only a few headers: - - [C] LIST HEADERS - [S] 215 headers supported: - [S] Subject - [S] Message-ID - [S] Xref - [S] . - - Example of an implementation providing access to the same fields as - the first example in Section 8.4.3: - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] OVER - [S] HDR - [S] LIST ACTIVE NEWSGROUPS HEADERS OVERVIEW.FMT - [S] . - [C] LIST HEADERS - [S] 215 headers and metadata items supported: - [S] Date - [S] Distribution - [S] From - [S] Message-ID - [S] References - [S] Subject - [S] Xref - [S] :bytes - [S] :lines - [S] . - - Example of an implementation providing access to all headers: - - [C] LIST HEADERS - [S] 215 metadata items supported: - [S] : - [S] :lines - [S] :bytes - [S] :x-article-number - [S] . - - - - - - - -Feather Standards Track [Page 89] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Example of an implementation distinguishing the first form of the HDR - command from the other two forms: - - [C] LIST HEADERS RANGE - [S] 215 metadata items supported: - [S] : - [S] :lines - [S] :bytes - [S] . - [C] LIST HEADERS MSGID - [S] 215 headers and metadata items supported: - [S] Date - [S] Distribution - [S] From - [S] Message-ID - [S] References - [S] Subject - [S] :lines - [S] :bytes - [S] :x-article-number - [S] . - [C] LIST HEADERS - [S] 215 headers and metadata items supported: - [S] Date - [S] Distribution - [S] From - [S] Message-ID - [S] References - [S] Subject - [S] :lines - [S] :bytes - [S] . - - Note that :x-article-number does not appear in the last set of - output. - -9. Augmented BNF Syntax for NNTP - -9.1. Introduction - - Each of the following sections describes the syntax of a major - element of NNTP. This syntax extends and refines the descriptions - elsewhere in this specification and should be given precedence when - resolving apparent conflicts. Note that ABNF [RFC4234] strings are - case insensitive. Non-terminals used in several places are defined - in a separate section at the end. - - - - - -Feather Standards Track [Page 90] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Between them, the non-terminals , , - , and specify the text that flows - between client and server. A consistent naming scheme is used in - this document for the non-terminals relating to each command, and - SHOULD be used by the specification of registered extensions. - - For each command, the sequence is as follows: - - o The client sends an instance of ; the syntax for the - EXAMPLE command is . - - o If the client is one that immediately streams data, it sends an - instance of ; the syntax for the EXAMPLE - command is . - - o The server sends an instance of . - - * The initial response line is independent of the command that - generated it; if the 000 response has arguments, the syntax of - the initial line is . - - * If the response is multi-line, the initial line is followed by - a . The syntax for the contents of this - block after "dot-stuffing" has been removed is (for the 000 - response to the EXAMPLE command) and - is an instance of . - - o While the latest response is one that indicates more data is - required (in general, a 3xx response): - - * the client sends an instance of ; the - syntax for the EXAMPLE continuation following a 333 response is - ; - - * the server sends another instance of , as above. - - (There are no commands in this specification that immediately stream - data, but this non-terminal is defined for the convenience of - extensions.) - - - - - - - - - - - - -Feather Standards Track [Page 91] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -9.2. Commands - - This syntax defines the non-terminal , which represents - what is sent from the client to the server (see section 3.1 for - limits on lengths). - - command-line = command EOL - command = X-command - X-command = keyword *(WS token) - - command =/ article-command / - body-command / - capabilities-command / - date-command / - group-command / - hdr-command / - head-command / - help-command / - ihave-command / - last-command / - list-command / - listgroup-command / - mode-reader-command / - newgroups-command / - newnews-command / - next-command / - over-command / - post-command / - quit-command / - stat-command - - article-command = "ARTICLE" [WS article-ref] - body-command = "BODY" [WS article-ref] - capabilities-command = "CAPABILITIES" [WS keyword] - date-command = "DATE" - group-command = "GROUP" [WS newsgroup-name] - hdr-command = "HDR" WS header-meta-name [WS range-ref] - head-command = "HEAD" [WS article-ref] - help-command = "HELP" - ihave-command = "IHAVE" WS message-id - last-command = "LAST" - list-command = "LIST" [WS list-arguments] - listgroup-command = "LISTGROUP" [WS newsgroup-name [WS range]] - mode-reader-command = "MODE" WS "READER" - newgroups-command = "NEWGROUPS" WS date-time - newnews-command = "NEWNEWS" WS wildmat WS date-time - next-command = "NEXT" - over-command = "OVER" [WS range-ref] - - - -Feather Standards Track [Page 92] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - post-command = "POST" - quit-command = "QUIT" - stat-command = "STAT" [WS article-ref] - - article-ref = article-number / message-id - date = date2y / date4y - date4y = 4DIGIT 2DIGIT 2DIGIT - date2y = 2DIGIT 2DIGIT 2DIGIT - date-time = date WS time [WS "GMT"] - header-meta-name = header-name / metadata-name - list-arguments = keyword [WS token] - metadata-name = ":" 1*A-NOTCOLON - range = article-number ["-" [article-number]] - range-ref = range / message-id - time = 2DIGIT 2DIGIT 2DIGIT - -9.3. Command Continuation - - This syntax defines the further material sent by the client in the - case of multi-stage commands and those that stream data. - - command-datastream = UNDEFINED - ; not used, provided as a hook for extensions - command-continuation = ihave-335-continuation / - post-340-continuation - - ihave-335-continuation = encoded-article - post-340-continuation = encoded-article - - encoded-article = multi-line-data-block - ; after undoing the "dot-stuffing", this MUST match
- -9.4. Responses - -9.4.1. Generic Responses - - This syntax defines the non-terminal , which represents the - generic form of responses; that is, what is sent from the server to - the client in response to a or a . - - response = simple-response / multi-line-response - simple-response = initial-response-line - multi-line-response = initial-response-line multi-line-data-block - - initial-response-line = - initial-response-content [SP trailing-comment] CRLF - initial-response-content = X-initial-response-content - X-initial-response-content = 3DIGIT *(SP response-argument) - - - -Feather Standards Track [Page 93] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - response-argument = 1*A-CHAR - trailing-comment = *U-CHAR - -9.4.2. Initial Response Line Contents - - This syntax defines the specific initial response lines for the - various commands in this specification (see section 3.1 for limits on - lengths). Only those response codes with arguments are listed. - - initial-response-content =/ response-111-content / - response-211-content / - response-220-content / - response-221-content / - response-222-content / - response-223-content / - response-401-content - - response-111-content = "111" SP date4y time - response-211-content = "211" 3(SP article-number) SP newsgroup-name - response-220-content = "220" SP article-number SP message-id - response-221-content = "221" SP article-number SP message-id - response-222-content = "222" SP article-number SP message-id - response-223-content = "223" SP article-number SP message-id - response-401-content = "401" SP capability-label - -9.4.3. Multi-line Response Contents - - This syntax defines the content of the various multi-line responses; - more precisely, it defines the part of the response in the multi-line - data block after any "dot-stuffing" has been undone. The numeric - portion of each non-terminal name indicates the response code that is - followed by this data. - - multi-line-response-content = article-220-ml-content / - body-222-ml-content / - capabilities-101-ml-content / - hdr-225-ml-content / - head-221-ml-content / - help-100-ml-content / - list-215-ml-content / - listgroup-211-ml-content / - newgroups-231-ml-content / - newnews-230-ml-content / - over-224-ml-content - - article-220-ml-content = article - body-222-ml-content = body - capabilities-101-ml-content = version-line CRLF - - - -Feather Standards Track [Page 94] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - *(capability-line CRLF) - hdr-225-ml-content = *(article-number SP hdr-content CRLF) - head-221-ml-content = 1*header - help-100-ml-content = *(*U-CHAR CRLF) - list-215-ml-content = list-content - listgroup-211-ml-content = *(article-number CRLF) - newgroups-231-ml-content = active-groups-list - newnews-230-ml-content = *(message-id CRLF) - over-224-ml-content = *(article-number over-content CRLF) - - active-groups-list = *(newsgroup-name SPA article-number - SPA article-number SPA newsgroup-status CRLF) - hdr-content = *S-NONTAB - hdr-n-content = [(header-name ":" / metadata-name) SP hdr-content] - list-content = body - newsgroup-status = %x79 / %x6E / %x6D / private-status - over-content = 1*6(TAB hdr-content) / - 7(TAB hdr-content) *(TAB hdr-n-content) - private-status = token ; except the values in newsgroup-status - -9.5. Capability Lines - - This syntax defines the generic form of a capability line in the - capabilities list (see Section 3.3.1). - - capability-line = capability-entry - capability-entry = X-capability-entry - X-capability-entry = capability-label *(WS capability-argument) - capability-label = keyword - capability-argument = token - - This syntax defines the specific capability entries for the - capabilities in this specification. - - capability-entry =/ - hdr-capability / - ihave-capability / - implementation-capability / - list-capability / - mode-reader-capability / - newnews-capability / - over-capability / - post-capability / - reader-capability - - hdr-capability = "HDR" - ihave-capability = "IHAVE" - implementation-capability = "IMPLEMENTATION" *(WS token) - - - -Feather Standards Track [Page 95] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - list-capability = "LIST" 1*(WS keyword) - mode-reader-capability = "MODE-READER" - newnews-capability = "NEWNEWS" - over-capability = "OVER" [WS "MSGID"] - post-capability = "POST" - reader-capability = "READER" - - version-line = "VERSION" 1*(WS version-number) - version-number = nzDIGIT *5DIGIT - -9.6. LIST Variants - - This section defines more specifically the keywords for the LIST - command and the syntax of the corresponding response contents. - - ; active - list-arguments =/ "ACTIVE" [WS wildmat] - list-content =/ list-active-content - list-active-content = active-groups-list - - - ; active.times - list-arguments =/ "ACTIVE.TIMES" [WS wildmat] - list-content =/ list-active-times-content - list-active-times-content = - *(newsgroup-name SPA 1*DIGIT SPA newsgroup-creator CRLF) - newsgroup-creator = U-TEXT - - - ; distrib.pats - list-arguments =/ "DISTRIB.PATS" - list-content =/ list-distrib-pats-content - list-distrib-pats-content = - *(1*DIGIT ":" wildmat ":" distribution CRLF) - distribution = token - - - ; headers - list-arguments =/ "HEADERS" [WS ("MSGID" / "RANGE")] - list-content =/ list-headers-content - list-headers-content = *(header-meta-name CRLF) / - *((metadata-name / ":") CRLF) - - - ; newsgroups - list-arguments =/ "NEWSGROUPS" [WS wildmat] - list-content =/ list-newsgroups-content - list-newsgroups-content = - - - -Feather Standards Track [Page 96] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - *(newsgroup-name WS newsgroup-description CRLF) - newsgroup-description = S-TEXT - - - ; overview.fmt - list-arguments =/ "OVERVIEW.FMT" - list-content =/ list-overview-fmt-content - list-overview-fmt-content = "Subject:" CRLF - "From:" CRLF - "Date:" CRLF - "Message-ID:" CRLF - "References:" CRLF - ( ":bytes" CRLF ":lines" / "Bytes:" CRLF "Lines:") CRLF - *((header-name ":full" / metadata-name) CRLF) - -9.7. Articles - - This syntax defines the non-terminal
, which represents the - format of an article as described in Section 3.6. - - article = 1*header CRLF body - header = header-name ":" [CRLF] SP header-content CRLF - header-content = *(S-CHAR / [CRLF] WS) - body = *(*B-CHAR CRLF) - -9.8. General Non-terminals - - These non-terminals are used at various places in the syntax and are - collected here for convenience. A few of these non-terminals are not - used in this specification but are provided for the consistency and - convenience of extension authors. - - multi-line-data-block = content-lines termination - content-lines = *([content-text] CRLF) - content-text = (".." / B-NONDOT) *B-CHAR - termination = "." CRLF - - article-number = 1*16DIGIT - header-name = 1*A-NOTCOLON - keyword = ALPHA 2*(ALPHA / DIGIT / "." / "-") - message-id = "<" 1*248A-NOTGT ">" - newsgroup-name = 1*wildmat-exact - token = 1*P-CHAR - - wildmat = wildmat-pattern *("," ["!"] wildmat-pattern) - wildmat-pattern = 1*wildmat-item - wildmat-item = wildmat-exact / wildmat-wild - wildmat-exact = %x22-29 / %x2B / %x2D-3E / %x40-5A / %x5E-7E / - - - -Feather Standards Track [Page 97] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - UTF8-non-ascii ; exclude ! * , ? [ \ ] - wildmat-wild = "*" / "?" - - base64 = *(4base64-char) [base64-terminal] - base64-char = UPPER / LOWER / DIGIT / "+" / "/" - base64-terminal = 2base64-char "==" / 3base64-char "=" - - ; Assorted special character sets - ; A- means based on US-ASCII, excluding controls and SP - ; P- means based on UTF-8, excluding controls and SP - ; U- means based on UTF-8, excluding NUL CR and LF - ; B- means based on bytes, excluding NUL CR and LF - A-CHAR = %x21-7E - A-NOTCOLON = %x21-39 / %x3B-7E ; exclude ":" - A-NOTGT = %x21-3D / %x3F-7E ; exclude ">" - P-CHAR = A-CHAR / UTF8-non-ascii - U-CHAR = CTRL / TAB / SP / A-CHAR / UTF8-non-ascii - U-NONTAB = CTRL / SP / A-CHAR / UTF8-non-ascii - U-TEXT = P-CHAR *U-CHAR - B-CHAR = CTRL / TAB / SP / %x21-FF - B-NONDOT = CTRL / TAB / SP / %x21-2D / %x2F-FF ; exclude "." - - ALPHA = UPPER / LOWER ; use only when case-insensitive - CR = %x0D - CRLF = CR LF - CTRL = %x01-08 / %x0B-0C / %x0E-1F - DIGIT = %x30-39 - nzDIGIT = %x31-39 - EOL = *(SP / TAB) CRLF - LF = %x0A - LOWER = %x61-7A - SP = %x20 - SPA = 1*SP - TAB = %x09 - UPPER = %x41-5A - UTF8-non-ascii = UTF8-2 / UTF8-3 / UTF8-4 - UTF8-2 = %xC2-DF UTF8-tail - UTF8-3 = %xE0 %xA0-BF UTF8-tail / %xE1-EC 2UTF8-tail / - %xED %x80-9F UTF8-tail / %xEE-EF 2UTF8-tail - UTF8-4 = %xF0 %x90-BF 2UTF8-tail / %xF1-F3 3UTF8-tail / - %xF4 %x80-8F 2UTF8-tail - UTF8-tail = %x80-BF - WS = 1*(SP / TAB) - - The following non-terminals require special consideration. They - represent situations where material SHOULD be restricted to UTF-8, - but implementations MUST be able to cope with other character - encodings. Therefore, there are two sets of definitions for them. - - - -Feather Standards Track [Page 98] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Implementations MUST accept any content that meets this syntax: - - S-CHAR = %x21-FF - S-NONTAB = CTRL / SP / S-CHAR - S-TEXT = (CTRL / S-CHAR) *B-CHAR - - and MAY pass such content on unaltered. - - When generating new content or re-encoding existing content, - implementations SHOULD conform to this syntax: - - S-CHAR = P-CHAR - S-NONTAB = U-NONTAB - S-TEXT = U-TEXT - -9.9. Extensions and Validation - - The specification of a registered extension MUST include formal - syntax that defines additional forms for the following non-terminals: - - command - for each new command other than a variant of the LIST command - - the syntax of each command MUST be compatible with the definition - of ; - - command-datastream - for each new command that immediately streams data; - - command-continuation - for each new command that sends further material after the initial - command line - the syntax of each continuation MUST be exactly - what is sent to the server, including any escape mechanisms such - as "dot-stuffing"; - - initial-response-content - for each new response code that has arguments - the syntax of each - response MUST be compatible with the definition of ; - - multi-line-response-content - for each new response code that has a multi-line response - the - syntax MUST show the response after the lines containing the - response code and the terminating octet have been removed and any - "dot-stuffing" undone; - - capability-entry - for each new capability label - the syntax of each entry MUST be - compatible with the definition of ; - - - -Feather Standards Track [Page 99] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - list-arguments - for each new variant of the LIST command - the syntax of each - entry MUST be compatible with the definition of ; - - list-content - for each new variant of the LIST command - the syntax MUST show - the response after the lines containing the 215 response code and - the terminating octet have been removed and any "dot-stuffing" - undone. - - The =/ notation of ABNF [RFC4234] and the naming conventions - described in Section 9.1 SHOULD be used for this. - - When the syntax in this specification, or syntax based on it, is - validated, it should be noted that: - - o the non-terminals , , - , , and - describe basic concepts of the - protocol and are not referred to by any other rule; - - o the non-terminal is provided for the convenience of - extension authors and is not referred to by any rule in this - specification; - - o for the reasons given above, the non-terminals , - , and each have two definitions; and - - o the non-terminal is deliberately not defined. - -10. Internationalisation Considerations - -10.1. Introduction and Historical Situation - - RFC 977 [RFC977] was written at a time when internationalisation was - not seen as a significant issue. As such, it was written on the - assumption that all communication would be in ASCII and use only a - 7-bit transport layer, although in practice just about all known - implementations are 8-bit clean. - - Since then, Usenet and NNTP have spread throughout the world. In the - absence of standards for handling the issues of language and - character sets, countries, newsgroup hierarchies, and individuals - have found a variety of solutions that work for them but that are not - necessarily appropriate elsewhere. For example, some have adopted a - default 8-bit character set appropriate to their needs (such as - ISO/IEC 8859-1 in Western Europe or KOI-8 in Russia), others have - used ASCII (either US-ASCII or national variants) in headers but - - - -Feather Standards Track [Page 100] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - local 16-bit character sets in article bodies, and still others have - gone for a combination of MIME [RFC2045] and UTF-8. With the - increased use of MIME in email, it is becoming more common to find - NNTP articles containing MIME headers that identify the character set - of the body, but this is far from universal. - - The resulting confusion does not help interoperability. - - One point that has been generally accepted is that articles can - contain octets with the top bit set, and NNTP is only expected to - operate on 8-bit clean transport paths. - -10.2. This Specification - - Part of the role of this present specification is to eliminate this - confusion and promote interoperability as far as possible. At the - same time, it is necessary to accept the existence of the present - situation and not break existing implementations and arrangements - gratuitously, even if they are less than optimal. Therefore, the - current practice described above has been taken into consideration in - producing this specification. - - This specification extends NNTP from US-ASCII [ANSI1986] to UTF-8 - [RFC3629]. Except in the two areas discussed below, UTF-8 (which is - a superset of US-ASCII) is mandatory, and implementations MUST NOT - use any other encoding. - - Firstly, the use of MIME for article headers and bodies is strongly - recommended. However, given widely divergent existing practices, an - attempt to require a particular encoding and tagging standard would - be premature at this time. Accordingly, this specification allows - the use of arbitrary 8-bit data in articles subject to the following - requirements and recommendations. - - o The names of headers (e.g., "From" or "Subject") MUST be in - US-ASCII. - - o Header values SHOULD use US-ASCII or an encoding based on it, such - as RFC 2047 [RFC2047], until such time as another approach has - been standardised. At present, 8-bit encodings (including UTF-8) - SHOULD NOT be used because they are likely to cause - interoperability problems. - - o The character set of article bodies SHOULD be indicated in the - article headers, and this SHOULD be done in accordance with MIME. - - o Where an article is obtained from an external source, an - implementation MAY pass it on and derive data from it (such as the - - - -Feather Standards Track [Page 101] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - response to the HDR command), even though the article or the data - does not meet the above requirements. Implementations MUST - transfer such articles and data correctly and unchanged; they MUST - NOT attempt to convert or re-encode the article or derived data. - (Nevertheless, a client or server MAY elect not to post or forward - the article if, after further examination of the article, it deems - it inappropriate to do so.) - - This requirement affects the ARTICLE (Section 6.2.1), BODY - (Section 6.2.3), HDR (Section 8.5), HEAD (Section 6.2.2), IHAVE - (Section 6.3.2), OVER (Section 8.3), and POST (Section 6.3.1) - commands. - - Secondly, the following requirements are placed on the newsgroups - list returned by the LIST NEWSGROUPS command (Section 7.6.6): - - o Although this specification allows UTF-8 for newsgroup names, they - SHOULD be restricted to US-ASCII until a successor to RFC 1036 - [RFC1036] standardises another approach. 8-bit encodings SHOULD - NOT be used because they are likely to cause interoperability - problems. - - o The newsgroup description SHOULD be in US-ASCII or UTF-8 unless - and until a successor to RFC 1036 standardises other encoding - arrangements. 8-bit encodings other than UTF-8 SHOULD NOT be used - because they are likely to cause interoperability problems. - - o Implementations that obtain this data from an external source MUST - handle it correctly even if it does not meet the above - requirements. Implementations (in particular, clients) MUST - handle such data correctly. - -10.3. Outstanding Issues - - While the primary use of NNTP is for transmitting articles that - conform to RFC 1036 (Netnews articles), it is also used for other - formats (see Appendix A). It is therefore most appropriate that - internationalisation issues related to article formats be addressed - in the relevant specifications. For Netnews articles, this is any - successor to RFC 1036. For email messages, it is RFC 2822 [RFC2822]. - - Of course, any article transmitted via NNTP needs to conform to this - specification as well. - - Restricting newsgroup names to UTF-8 is not a complete solution. In - particular, when new newsgroup names are created or a user is asked - to enter a newsgroup name, some scheme of canonicalisation will need - to take place. This specification does not attempt to define that - - - -Feather Standards Track [Page 102] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - canonicalization; further work is needed in this area, in conjunction - with the article format specifications. Until such specifications - are published, implementations SHOULD match newsgroup names octet by - octet. It is anticipated that any approved scheme will be applied - "at the edges", and therefore octet-by-octet comparison will continue - to apply to most, if not all, uses of newsgroup names in NNTP. - - In the meantime, any implementation experimenting with UTF-8 - newsgroup names is strongly cautioned that a future specification may - require that those names be canonicalized when used with NNTP in a - way that is not compatible with their experiments. - - Since the primary use of NNTP is with Netnews, and since newsgroup - descriptions are normally distributed through specially formatted - articles, it is recommended that the internationalisation issues - related to them be addressed in any successor to RFC 1036. - -11. IANA Considerations - - This specification requires IANA to keep a registry of capability - labels. The initial contents of this registry are specified in - Section 3.3.4. As described in Section 3.3.3, labels beginning with - X are reserved for private use, while all other names are expected to - be associated with a specification in an RFC on the standards track - or defining an IESG-approved experimental protocol. - - Different entries in the registry MUST use different capability - labels. - - Different entries in the registry MUST NOT use the same command name. - For this purpose, variants distinguished by a second or subsequent - keyword (e.g., "LIST HEADERS" and "LIST OVERVIEW.FMT") count as - different commands. If there is a need for two extensions to use the - same command, a single harmonised specification MUST be registered. - -12. Security Considerations - - This section is meant to inform application developers, information - providers, and users of the security limitations in NNTP as described - by this document. The discussion does not include definitive - solutions to the problems revealed, though it does make some - suggestions for reducing security risks. - - - - - - - - - -Feather Standards Track [Page 103] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -12.1. Personal and Proprietary Information - - NNTP, because it was created to distribute network news articles, - will forward whatever information is stored in those articles. - Specification of that information is outside this scope of this - document, but it is likely that some personal and/or proprietary - information is available in some of those articles. It is very - important that designers and implementers provide informative - warnings to users so that personal and/or proprietary information in - material that is added automatically to articles (e.g., in headers) - is not disclosed inadvertently. Additionally, effective and easily - understood mechanisms to manage the distribution of news articles - SHOULD be provided to NNTP Server administrators, so that they are - able to report with confidence the likely spread of any particular - set of news articles. - -12.2. Abuse of Server Log Information - - A server is in the position to save session data about a user's - requests that might identify their reading patterns or subjects of - interest. This information is clearly confidential in nature, and - its handling can be constrained by law in certain countries. People - using this protocol to provide data are responsible for ensuring that - such material is not distributed without the permission of any - individuals that are identifiable by the published results. - -12.3. Weak Authentication and Access Control - - There is no user-based or token-based authentication in the basic - NNTP specification. Access is normally controlled by server - configuration files. Those files specify access by using domain - names or IP addresses. However, this specification does permit the - creation of extensions to NNTP for such purposes; one such extension - is [NNTP-AUTH]. While including such mechanisms is optional, doing - so is strongly encouraged. - - Other mechanisms are also available. For example, a proxy server - could be put in place that requires authentication before connecting - via the proxy to the NNTP server. - -12.4. DNS Spoofing - - Many existing NNTP implementations authorize incoming connections by - checking the IP address of that connection against the IP addresses - obtained via DNS lookups of lists of domain names given in local - configuration files. Servers that use this type of authentication - and clients that find a server by doing a DNS lookup of the server - name rely very heavily on the Domain Name Service, and are thus - - - -Feather Standards Track [Page 104] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - generally prone to security attacks based on the deliberate - misassociation of IP addresses and DNS names. Clients and servers - need to be cautious in assuming the continuing validity of an IP - number/DNS name association. - - In particular, NNTP clients and servers SHOULD rely on their name - resolver for confirmation of an IP number/DNS name association, - rather than cache the result of previous host name lookups. Many - platforms already can cache host name lookups locally when - appropriate, and they SHOULD be configured to do so. It is proper - for these lookups to be cached, however, only when the TTL (Time To - Live) information reported by the name server makes it likely that - the cached information will remain useful. - - If NNTP clients or servers cache the results of host name lookups in - order to achieve a performance improvement, they MUST observe the TTL - information reported by DNS. If NNTP clients or servers do not - observe this rule, they could be spoofed when a previously accessed - server's IP address changes. As network renumbering is expected to - become increasingly common, the possibility of this form of attack - will increase. Observing this requirement thus reduces this - potential security vulnerability. - - This requirement also improves the load-balancing behaviour of - clients for replicated servers using the same DNS name and reduces - the likelihood of a user's experiencing failure in accessing sites - that use that strategy. - -12.5. UTF-8 Issues - - UTF-8 [RFC3629] permits only certain sequences of octets and - designates others as either malformed or "illegal". The Unicode - standard identifies a number of security issues related to illegal - sequences and forbids their generation by conforming implementations. - - Implementations of this specification MUST NOT generate malformed or - illegal sequences and SHOULD detect them and take some appropriate - action. This could include the following: - - o Generating a 501 response code. - - o Replacing such sequences by the sequence %xEF.BF.BD, which encodes - the "replacement character" U+FFFD. - - o Closing the connection. - - o Replacing such sequences by a "guessed" valid sequence (based on - properties of the UTF-8 encoding). - - - -Feather Standards Track [Page 105] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - In the last case, the implementation MUST ensure that any replacement - cannot be used to bypass validity or security checks. For example, - the illegal sequence %xC0.A0 is an over-long encoding for space - (%x20). If it is replaced by the correct encoding in a command line, - this needs to happen before the command line is parsed into - individual arguments. If the replacement came after parsing, it - would be possible to generate an argument with an embedded space, - which is forbidden. Use of the "replacement character" does not have - this problem, since it is permitted wherever non-US-ASCII characters - are. Implementations SHOULD use one of the first two solutions where - the general structure of the NNTP stream remains intact and SHOULD - close the connection if it is no longer possible to parse it - sensibly. - -12.6. Caching of Capability Lists - - The CAPABILITIES command provides a capability list, which is - information about the current capabilities of the server. Whenever - there is a relevant change to the server state, the results of this - command are required to change accordingly. - - In most situations, the capabilities list in a given server state - will not change from session to session; for example, a given - extension will be installed permanently on a server. Some clients - may therefore wish to remember which extensions a server supports to - avoid the delay of an additional command and response, particularly - if they open multiple connections in the same session. - - However, information about extensions related to security and privacy - MUST NOT be cached, since this could allow a variety of attacks. - - For example, consider a server that permits the use of cleartext - passwords on links that are encrypted but not otherwise: - - [Initial connection set-up completed.] - [S] 200 NNTP Service Ready, posting permitted - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] NEWNEWS - [S] POST - [S] XENCRYPT - [S] LIST ACTIVE NEWSGROUPS - [S] . - [C] XENCRYPT - [Client and server negotiate encryption on the link] - [S] 283 Encrypted link established - - - -Feather Standards Track [Page 106] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - [C] CAPABILITIES - [S] 101 Capability list: - [S] VERSION 2 - [S] READER - [S] NEWNEWS - [S] POST - [S] XSECRET - [S] LIST ACTIVE NEWSGROUPS - [S] . - [C] XSECRET fred flintstone - [S] 290 Password for fred accepted - - If the client caches the last capabilities list, then on the next - session it will attempt to use XSECRET on an unencrypted link: - - [Initial connection set-up completed.] - [S] 200 NNTP Service Ready, posting permitted - [C] XSECRET fred flintstone - [S] 483 Only permitted on secure links - - This exposes the password to any eavesdropper. While the primary - cause of this is passing a secret without first checking the security - of the link, caching of capability lists can increase the risk. - - Any security extension should include requirements to check the - security state of the link in a manner appropriate to that extension. - - Caching should normally only be considered for anonymous clients that - do not use any security or privacy extensions and for which the time - required for an additional command and response is a noticeable - issue. - -13. Acknowledgements - - This document is the result of much effort by the present and past - members of the NNTP Working Group, chaired by Russ Allbery and Ned - Freed. It could not have been produced without them. - - The author acknowledges the original authors of NNTP as documented in - RFC 977 [RFC977]: Brian Kantor and Phil Lapsey. - - The author gratefully acknowledges the following: - - o The work of the NNTP committee chaired by Eliot Lear. The - organization of this document was influenced by the last available - version from this working group. A special thanks to Eliot for - generously providing the original machine-readable sources for - that document. - - - -Feather Standards Track [Page 107] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - o The work of the DRUMS working group, specifically RFC 1869 - [RFC1869], that drove the original thinking that led to the - CAPABILITIES command and the extensions mechanism detailed in this - document. - - o The authors of RFC 2616 [RFC2616] for providing specific and - relevant examples of security issues that should be considered for - HTTP. Since many of the same considerations exist for NNTP, those - examples that are relevant have been included here with some minor - rewrites. - - o The comments and additional information provided by the following - individuals in preparing one or more of the progenitors of this - document: - - Russ Allbery - Wayne Davison - Chris Lewis - Tom Limoncelli - Eric Schnoebelen - Rich Salz - - This work was motivated by the work of various news reader authors - and news server authors, including those listed below: - - Rick Adams - Original author of the NNTP extensions to the RN news reader and - last maintainer of Bnews. - - Stan Barber - Original author of the NNTP extensions to the news readers that - are part of Bnews. - - Geoff Collyer - Original author of the OVERVIEW database proposal and one of the - original authors of CNEWS. - - Dan Curry - Original author of the xvnews news reader. - - Wayne Davison - Author of the first threading extensions to the RN news reader - (commonly called TRN). - - Geoff Huston - Original author of ANU NEWS. - - - - - -Feather Standards Track [Page 108] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Phil Lapsey - Original author of the UNIX reference implementation for NNTP. - - Iain Lea - Original maintainer of the TIN news reader. - - Chris Lewis - First known implementer of the AUTHINFO GENERIC extension. - - Rich Salz - Original author of INN. - - Henry Spencer - One of the original authors of CNEWS. - - Kim Storm - Original author of the NN news reader. - - Other people who contributed to this document include: - - Matthias Andree - Greg Andruk - Daniel Barclay - Maurizio Codogno - Mark Crispin - Andrew Gierth - Juergen Helbing - Scott Hollenbeck - Urs Janssen - Charles Lindsey - Ade Lovett - David Magda - Ken Murchison - Francois Petillon - Peter Robinson - Rob Siemborski - Howard Swinehart - Ruud van Tol - Jeffrey Vinocur - Erik Warmelink - - The author thanks them all and apologises to anyone omitted. - - Finally, the present author gratefully acknowledges the vast amount - of work put into previous versions by the previous author: - - Stan Barber - - - - -Feather Standards Track [Page 109] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -14. References - -14.1. Normative References - - [ANSI1986] American National Standards Institute, "Coded Character - Set - 7-bit American Standard Code for Information - Interchange", ANSI X3.4, 1986. - - [RFC977] Kantor, B. and P. Lapsley, "Network News Transfer - Protocol", RFC 977, February 1986. - - [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet - Mail Extensions (MIME) Part One: Format of Internet - Message Bodies", RFC 2045, November 1996. - - [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail - Extensions) Part Three: Message Header Extensions for - Non-ASCII Text", RFC 2047, November 1996. - - [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate - Requirement Levels", BCP 14, RFC 2119, March 1997. - - [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO - 10646", STD 63, RFC 3629, November 2003. - - [RFC4234] Crocker, D., Ed. and P. Overell, "Augmented BNF for - Syntax Specifications: ABNF", RFC 4234, October 2005. - - [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data - Encodings", RFC 4648, October 2006. - - [TF.686-1] International Telecommunications Union - Radio, - "Glossary, ITU-R Recommendation TF.686-1", - ITU-R Recommendation TF.686-1, October 1997. - -14.2. Informative References - - [NNTP-AUTH] Vinocur, J., Murchison, K., and C. Newman, "Network - News Transfer Protocol (NNTP) Extension for - Authentication", - RFC 4643, October 2006. - - [NNTP-STREAM] Vinocur, J. and K. Murchison, "Network News Transfer - Protocol (NNTP) Extension for Streaming Feeds", - RFC 4644, October 2006. - - - - - - -Feather Standards Track [Page 110] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - [NNTP-TLS] Murchison, K., Vinocur, J., and C. Newman, "Using - Transport Layer Security (TLS) with Network News - Transfer Protocol (NNTP)", RFC 4642, October 2006. - - [RFC1036] Horton, M. and R. Adams, "Standard for interchange of - USENET messages", RFC 1036, December 1987. - - [RFC1305] Mills, D., "Network Time Protocol (Version 3) - Specification, Implementation and Analysis", RFC 1305, - March 1992. - - [RFC1869] Klensin, J., Freed, N., Rose, M., Stefferud, E., and D. - Crocker, "SMTP Service Extensions", STD 10, RFC 1869, - November 1995. - - [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., - Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext - Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. - - [RFC2629] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, - June 1999. - - [RFC2822] Resnick, P., "Internet Message Format", RFC 2822, April - 2001. - - [RFC2980] Barber, S., "Common NNTP Extensions", RFC 2980, October - 2000. - - [ROBE1995] Robertson, R., "FAQ: Overview database / NOV General - Information", January 1995. - - There is no definitive copy of this document known to - the author. It was previously posted as the Usenet - article - - [SALZ1992] Salz, R., "Manual Page for wildmat(3) from the INN 1.4 - distribution, Revision 1.10", April 1992. - - There is no definitive copy of this document known to - the author. - - - - - - - - - - - -Feather Standards Track [Page 111] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -Appendix A. Interaction with Other Specifications - - NNTP is most often used for transferring articles that conform to - RFC 1036 [RFC1036] (such articles are called "Netnews articles" - here). It is also sometimes used for transferring email messages - that conform to RFC 2822 [RFC2822] (such articles are called "email - articles" here). In this situation, articles must conform both to - this specification and to that other one; this appendix describes - some relevant issues. - -A.1. Header Folding - - NNTP allows a header line to be folded (by inserting a CRLF pair) - before any space or TAB character. - - Both email and Netnews articles are required to have at least one - octet other than space or TAB on each header line. Thus, folding can - only happen at one point in each sequence of consecutive spaces or - TABs. Netnews articles are further required to have the header name, - colon, and following space all on the first line; folding may only - happen beyond that space. Finally, some non-conforming software will - remove trailing spaces and TABs from a line. Therefore, it might be - inadvisable to fold a header after a space or TAB. - - For maximum safety, header lines SHOULD conform to the following - syntax rather than to that in Section 9.7. - - - header = header-name ":" SP [header-content] CRLF - header-content = [WS] token *( [CRLF] WS token ) - -A.2. Message-IDs - - Every article handled by an NNTP server MUST have a unique - message-id. For the purposes of this specification, a message-id is - an arbitrary opaque string that merely needs to meet certain - syntactic requirements and is just a way to refer to the article. - - Because there is a significant risk that old articles will be - reinjected into the global Usenet system, RFC 1036 [RFC1036] requires - that message-ids are globally unique for all time. - - This specification states that message-ids are the same if and only - if they consist of the same sequence of octets. Other specifications - may define two different sequences as being equal because they are - putting an interpretation on particular characters. RFC 2822 - [RFC2822] has a concept of "quoted" and "escaped" characters. It - therefore considers the three message-ids: - - - -Feather Standards Track [Page 112] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - - <"ab.cd"@example.com> - <"ab.\cd"@example.com> - - as being identical. Therefore, an NNTP implementation handing email - articles must ensure that only one of these three appears in the - protocol and that the other two are converted to it as and when - necessary, such as when a client checks the results of a NEWNEWS - command against an internal database of message-ids. Note that - RFC 1036 [RFC1036] never treats two different strings as being - identical. Its successor (as of the time of writing) restricts the - syntax of message-ids so that, whenever RFC 2822 would treat two - strings as equivalent, only one of them is valid (in the above - example, only the first string is valid). - - This specification does not describe how the message-id of an article - is determined; it may be deduced from the contents of the article or - derived from some external source. If the server is also conforming - to another specification that contains a definition of message-id - compatible with this one, the server SHOULD use those message-ids. A - common approach, and one that SHOULD be used for email and Netnews - articles, is to extract the message-id from the contents of a header - with name "Message-ID". This may not be as simple as copying the - entire header contents; it may be necessary to strip off comments and - undo quoting, or to reduce "equivalent" message-ids to a canonical - form. - - If an article is obtained through the IHAVE command, there will be a - message-id provided with the command. The server MAY either use it - or determine one from the article contents. However, whichever it - does, it SHOULD ensure that, if the IHAVE command is repeated with - the same argument and article, it will be recognized as a duplicate. - - If an article does not contain a message-id that the server can - identify, it MUST synthesize one. This could, for example, be a - simple sequence number or be based on the date and time when the - article arrived. When email or Netnews articles are handled, a - Message-ID header SHOULD be added to ensure global consistency and - uniqueness. - - Note that, because the message-id might not have been derived from - the Message-ID header in the article, the following example is - legitimate (though unusual): - - - - - - - - -Feather Standards Track [Page 113] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - [C] HEAD <45223423@example.com> - [S] 221 0 <45223423@example.com> - [S] Path: pathost!demo!whitehouse!not-for-mail - [S] Message-ID: <1234@example.net> - [S] From: "Demo User" - [S] Newsgroups: misc.test - [S] Subject: I am just a test article - [S] Date: 6 Oct 1998 04:38:40 -0500 - [S] Organization: An Example Net, Uncertain, Texas - [S] . - -A.3. Article Posting - - As far as NNTP is concerned, the POST and IHAVE commands provide the - same basic facilities in a slightly different way. However, they - have rather different intentions. - - The IHAVE command is intended for transmitting conforming articles - between a system of NNTP servers, with all articles perhaps also - conforming to another specification (e.g., all articles are Netnews - articles). It is expected that the client will already have done any - necessary validation (or that it has in turn obtained the article - from a third party that has done so); therefore, the contents SHOULD - be left unchanged. - - In contrast, the POST command is intended for use when an end-user is - injecting a newly created article into a such a system. The article - being transferred might not be a conforming email or Netnews article, - and the server is expected to validate it and, if necessary, to - convert it to the right form for onward distribution. This is often - done by a separate piece of software on the server installation; if - so, the NNTP server SHOULD pass the incoming article to that software - unaltered, making no attempt to filter characters, to fold or limit - lines, or to process the incoming text otherwise. - - The POST command can fail in various ways, and clients should be - prepared to re-send an article. When doing so, however, it is often - important to ensure (as far as possible) that the same message-id is - allocated to both attempts so that the server, or other servers, can - recognize the two articles as duplicates. In the case of email or - Netnews articles, therefore, the posted article SHOULD contain a - header with the name "Message-ID", and the contents of this header - SHOULD be identical on each attempt. The server SHOULD ensure that - two POSTed articles with the same contents for this header are - recognized as identical and that the same message-id is allocated, - whether or not those contents are suitable for use as the message-id. - - - - - -Feather Standards Track [Page 114] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -Appendix B. Summary of Commands - - This section contains a list of every command defined in this - document, ordered by command name and by indicating capability. - - Ordered by command name: - - +-------------------+-----------------------+---------------+ - | Command | Indicating capability | Definition | - +-------------------+-----------------------+---------------+ - | ARTICLE | READER | Section 6.2.1 | - | BODY | READER | Section 6.2.3 | - | CAPABILITIES | mandatory | Section 5.2 | - | DATE | READER | Section 7.1 | - | GROUP | READER | Section 6.1.1 | - | HDR | HDR | Section 8.5 | - | HEAD | mandatory | Section 6.2.2 | - | HELP | mandatory | Section 7.2 | - | IHAVE | IHAVE | Section 6.3.2 | - | LAST | READER | Section 6.1.3 | - | LIST | LIST | Section 7.6.1 | - | LIST ACTIVE.TIMES | LIST | Section 7.6.4 | - | LIST ACTIVE | LIST | Section 7.6.3 | - | LIST DISTRIB.PATS | LIST | Section 7.6.5 | - | LIST HEADERS | HDR | Section 8.6 | - | LIST NEWSGROUPS | LIST | Section 7.6.6 | - | LIST OVERVIEW.FMT | OVER | Section 8.4 | - | LISTGROUP | READER | Section 6.1.2 | - | MODE READER | MODE-READER | Section 5.3 | - | NEWGROUPS | READER | Section 7.3 | - | NEWNEWS | NEWNEWS | Section 7.4 | - | NEXT | READER | Section 6.1.4 | - | OVER | OVER | Section 8.3 | - | POST | POST | Section 6.3.1 | - | QUIT | mandatory | Section 5.4 | - | STAT | mandatory | Section 6.2.4 | - +-------------------+-----------------------+---------------+ - - - - - - - - - - - - - - -Feather Standards Track [Page 115] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Ordered by indicating capability: - - +-------------------+-----------------------+---------------+ - | Command | Indicating capability | Definition | - +-------------------+-----------------------+---------------+ - | CAPABILITIES | mandatory | Section 5.2 | - | HEAD | mandatory | Section 6.2.2 | - | HELP | mandatory | Section 7.2 | - | QUIT | mandatory | Section 5.4 | - | STAT | mandatory | Section 6.2.4 | - | HDR | HDR | Section 8.5 | - | LIST HEADERS | HDR | Section 8.6 | - | IHAVE | IHAVE | Section 6.3.2 | - | LIST | LIST | Section 7.6.1 | - | LIST ACTIVE | LIST | Section 7.6.3 | - | LIST ACTIVE.TIMES | LIST | Section 7.6.4 | - | LIST DISTRIB.PATS | LIST | Section 7.6.5 | - | LIST NEWSGROUPS | LIST | Section 7.6.6 | - | MODE READER | MODE-READER | Section 5.3 | - | NEWNEWS | NEWNEWS | Section 7.4 | - | OVER | OVER | Section 8.3 | - | LIST OVERVIEW.FMT | OVER | Section 8.4 | - | POST | POST | Section 6.3.1 | - | ARTICLE | READER | Section 6.2.1 | - | BODY | READER | Section 6.2.3 | - | DATE | READER | Section 7.1 | - | GROUP | READER | Section 6.1.1 | - | LAST | READER | Section 6.1.3 | - | LISTGROUP | READER | Section 6.1.2 | - | NEWGROUPS | READER | Section 7.3 | - | NEXT | READER | Section 6.1.4 | - +-------------------+-----------------------+---------------+ - - - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 116] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -Appendix C. Summary of Response Codes - - This section contains a list of every response code defined in this - document and indicates whether it is multi-line, which commands can - generate it, what arguments it has, and what its meaning is. - - Response code 100 (multi-line) - Generated by: HELP - Meaning: help text follows. - - Response code 101 (multi-line) - Generated by: CAPABILITIES - Meaning: capabilities list follows. - - Response code 111 - Generated by: DATE - 1 argument: yyyymmddhhmmss - Meaning: server date and time. - - Response code 200 - Generated by: initial connection, MODE READER - Meaning: service available, posting allowed. - - Response code 201 - Generated by: initial connection, MODE READER - Meaning: service available, posting prohibited. - - Response code 205 - Generated by: QUIT - Meaning: connection closing (the server immediately closes the - connection). - - Response code 211 - The 211 response code has two completely different forms, - depending on which command generated it: - - (not multi-line) - Generated by: GROUP - 4 arguments: number low high group - Meaning: group selected. - - (multi-line) - Generated by: LISTGROUP - 4 arguments: number low high group - Meaning: article numbers follow. - - - - - - -Feather Standards Track [Page 117] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Response code 215 (multi-line) - Generated by: LIST - Meaning: information follows. - - Response code 220 (multi-line) - Generated by: ARTICLE - 2 arguments: n message-id - Meaning: article follows. - - Response code 221 (multi-line) - Generated by: HEAD - 2 arguments: n message-id - Meaning: article headers follow. - - Response code 222 (multi-line) - Generated by: BODY - 2 arguments: n message-id - Meaning: article body follows. - - Response code 223 - Generated by: LAST, NEXT, STAT - 2 arguments: n message-id - Meaning: article exists and selected. - - Response code 224 (multi-line) - Generated by: OVER - Meaning: overview information follows. - - Response code 225 (multi-line) - Generated by: HDR - Meaning: headers follow. - - Response code 230 (multi-line) - Generated by: NEWNEWS - Meaning: list of new articles follows. - - Response code 231 (multi-line) - Generated by: NEWGROUPS - Meaning: list of new newsgroups follows. - - Response code 235 - Generated by: IHAVE (second stage) - Meaning: article transferred OK. - - Response code 240 - Generated by: POST (second stage) - Meaning: article received OK. - - - - -Feather Standards Track [Page 118] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Response code 335 - Generated by: IHAVE (first stage) - Meaning: send article to be transferred. - - Response code 340 - Generated by: POST (first stage) - Meaning: send article to be posted. - - Response code 400 - Generic response and generated by initial connection - Meaning: service not available or no longer available (the server - immediately closes the connection). - - Response code 401 - Generic response - 1 argument: capability-label - Meaning: the server is in the wrong mode; the indicated capability - should be used to change the mode. - - Response code 403 - Generic response - Meaning: internal fault or problem preventing action being taken. - - Response code 411 - Generated by: GROUP, LISTGROUP - Meaning: no such newsgroup. - - Response code 412 - Generated by: ARTICLE, BODY, GROUP, HDR, HEAD, LAST, LISTGROUP, - NEXT, OVER, STAT - Meaning: no newsgroup selected. - - Response code 420 - Generated by: ARTICLE, BODY, HDR, HEAD, LAST, NEXT, OVER, STAT - Meaning: current article number is invalid. - - Response code 421 - Generated by: NEXT - Meaning: no next article in this group. - - Response code 422 - Generated by: LAST - Meaning: no previous article in this group. - - Response code 423 - Generated by: ARTICLE, BODY, HDR, HEAD, OVER, STAT - Meaning: no article with that number or in that range. - - - - -Feather Standards Track [Page 119] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Response code 430 - Generated by: ARTICLE, BODY, HDR, HEAD, OVER, STAT - Meaning: no article with that message-id. - - Response code 435 - Generated by: IHAVE (first stage) - Meaning: article not wanted. - - Response code 436 - Generated by: IHAVE (either stage) - Meaning: transfer not possible (first stage) or failed (second - stage); try again later. - - Response code 437 - Generated by: IHAVE (second stage) - Meaning: transfer rejected; do not retry. - - Response code 440 - Generated by: POST (first stage) - Meaning: posting not permitted. - - Response code 441 - Generated by: POST (second stage) - Meaning: posting failed. - - Response code 480 - Generic response - Meaning: command unavailable until the client has authenticated - itself. - - Response code 483 - Generic response - Meaning: command unavailable until suitable privacy has been - arranged. - - Response code 500 - Generic response - Meaning: unknown command. - - Response code 501 - Generic response - Meaning: syntax error in command. - - - - - - - - - -Feather Standards Track [Page 120] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - Response code 502 - Generic response and generated by initial connection - - Meaning for the initial connection and the MODE READER command: - service permanently unavailable (the server immediately closes the - connection). - - Meaning for all other commands: command not permitted (and there - is no way for the client to change this). - - Response code 503 - Generic response - Meaning: feature not supported. - - Response code 504 - Generic response - Meaning: error in base64-encoding [RFC4648] of an argument. - -Appendix D. Changes from RFC 977 - - In general every attempt has been made to ensure that the protocol - specification in this document is compatible with the version - specified in RFC 977 [RFC977] and the various facilities adopted from - RFC 2980 [RFC2980]. However, there have been a number of changes, - some compatible and some not. - - This appendix lists these changes. It is not guaranteed to be - exhaustive or correct and MUST NOT be relied on. - - o A formal syntax specification (Section 9) has been added. - - o The default character set is changed from US-ASCII [ANSI1986] to - UTF-8 [RFC3629] (note that US-ASCII is a subset of UTF-8). This - matter is discussed further in Section 10. - - o All articles are required to have a message-id, eliminating the - "<0>" placeholder used in RFC 977 in some responses. - - o The newsgroup name matching capabilities already documented in - RFC 977 ("wildmats", Section 4) are clarified and extended. The - new facilities (e.g., the use of commas and exclamation marks) are - allowed wherever wildmats appear in the protocol. - - o Support for pipelining of commands (Section 3.5) is made - mandatory. - - - - - - -Feather Standards Track [Page 121] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - o The principles behind response codes (Section 3.2) have been - tidied up. In particular: - - * the x8x response code family, formerly used for private - extensions, is now reserved for authentication and privacy - extensions; - - * the x9x response code family, formerly intended for debugging - facilities, are now reserved for private extensions; - - * the 502 and 503 generic response codes (Section 3.2.1) have - been redefined; - - * new 401, 403, 480, 483, and 504 generic response codes have - been added. - - o The rules for article numbering (Section 6) have been clarified - (also see Section 6.1.1.2). - - o The SLAVE command (which was ill-defined) is removed from the - protocol. - - o Four-digit years are permitted in the NEWNEWS (Section 7.4) and - NEWGROUPS (Section 7.3) commands (two-digit years are still - permitted). The optional distribution parameter to these commands - has been removed. - - o The LIST command (Section 7.6.1) is greatly extended; the original - is available as LIST ACTIVE, while new variants include - ACTIVE.TIMES, DISTRIB.PATS, and NEWSGROUPS. A new "m" status flag - is added to the LIST ACTIVE response. - - o A new CAPABILITIES command (Section 5.2) allows clients to - determine what facilities are supported by a server. - - o The DATE command (Section 7.1) is adopted from RFC 2980 - effectively unchanged. - - o The LISTGROUP command (Section 6.1.2) is adopted from RFC 2980. - An optional range argument has been added, and the 211 initial - response line now has the same format as the 211 response from the - GROUP command. - - o The MODE READER command (Section 5.3) is adopted from RFC 2980 and - its meaning and effects clarified. - - o The XHDR command in RFC 2980 has been formalised as the new HDR - (Section 8.5) and LIST HEADERS (Section 8.6) commands. - - - -Feather Standards Track [Page 122] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - - o The XOVER command in RFC 2980 has been formalised as the new OVER - (Section 8.3) and LIST OVERVIEW.FMT (Section 8.4) commands. The - former can be applied to a message-id as well as to a range. - - o The concept of article metadata (Section 8.1) has been formalised, - allowing the Bytes and Lines pseudo-headers to be deprecated. - - Client authors should note in particular that lack of support for the - CAPABILITIES command is a good indication that the server does not - support this specification. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 123] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -Author's Address - - Clive D.W. Feather - THUS plc - 322 Regents Park Road - London - N3 2QQ - United Kingdom - - Phone: +44 20 8495 6138 - Fax: +44 870 051 9937 - EMail: clive@demon.net - URI: http://www.davros.org/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Feather Standards Track [Page 124] - -RFC 3977 Network News Transfer Protocol (NNTP) October 2006 - - -Full Copyright Statement - -Copyright (C) The Internet Society (2006). - - This document is subject to the rights, licenses and restrictions - contained in BCP 78, and except as set forth therein, the authors - retain all their rights. - - This document and the information contained herein are provided on an - "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS - OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET - ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, - INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE - INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED - WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. - -Intellectual Property - - The IETF takes no position regarding the validity or scope of any - Intellectual Property Rights or other rights that might be claimed to - pertain to the implementation or use of the technology described in - this document or the extent to which any license under such rights - might or might not be available; nor does it represent that it has - made any independent effort to identify any such rights. Information - on the procedures with respect to rights in RFC documents can be - found in BCP 78 and BCP 79. - - Copies of IPR disclosures made to the IETF Secretariat and any - assurances of licenses to be made available, or the result of an - attempt made to obtain a general license or permission for the use of - such proprietary rights by implementers or users of this - specification can be obtained from the IETF on-line IPR repository at - http://www.ietf.org/ipr. - - The IETF invites any interested party to bring to its attention any - copyrights, patents or patent applications, or other proprietary - rights that may cover technology that may be required to implement - this standard. Please address the information to the IETF at ietf- - ipr@ietf.org. - -Acknowledgement - - Funding for the RFC Editor function is provided by the IETF - Administrative Support Activity (IASA). - - - - - - - -Feather Standards Track [Page 125] - diff -r f907866f0e4b -r 6fceb66e1ad7 trunk/test/StringTemplateTest.java --- a/trunk/test/StringTemplateTest.java Tue Jan 20 10:21:03 2009 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,26 +0,0 @@ -/* - * To change this template, choose Tools | Templates - * and open the template in the editor. - */ - -package test; - -import com.so.news.util.StringTemplate; - -/** - * - * @author chris - */ -public class StringTemplateTest -{ - public static void main(String[] args) - { - StringTemplate templ - = new StringTemplate("SELECT %row FROM %table WHERE %row = ich"); - - templ.set("row", "name"); - templ.set("table", "UserTable"); - - System.out.println(templ.toString()); - } -}