# HOW TO EDIT THIS FILE: # The "handy ruler" below makes it easier to edit a package description. # Line up the first '|' above the ':' following the base package name, and # the '|' on the right side marks the last column you can put a character in. # You must make exactly 11 lines for the formatting to be correct. It's also # customary to leave one space after the ':' except on otherwise blank lines. |-----handy-ruler------------------------------------------------------| perl-www-robotrules: perl-www-robotrules (DB of robots.txt-derived perms) perl-www-robotrules: perl-www-robotrules: This perl module parses /robots.txt files as specified in "A perl-www-robotrules: Standard for Robot Exclusion". Webmasters can use the /robots.txt perl-www-robotrules: file to forbid conforming robots from accessing parts of their perl-www-robotrules: web site perl-www-robotrules: perl-www-robotrules: https://metacpan.org/release/WWW-RobotRules perl-www-robotrules: perl-www-robotrules: perl-www-robotrules: