DESCRIPTION

Package: spiderbar
Type: Package
Title: Parse and Test Robots Exclusion Protocol Files and Rules
Version: 0.2.4
Date: 2021-05-14
Author: Bob Rudis (bob@rud.is) [aut, cre], SEOmoz, Inc [aut]
Maintainer: Bob Rudis <bob@rud.is>
Description: The 'Robots Exclusion Protocol' <https://www.robotstxt.org/orig.html> documents
    a set of standards for allowing or excluding robot/spider crawling of different areas of
    site content. Tools are provided which wrap The 'rep-cpp' <https://github.com/seomoz/rep-cpp>
    C++ library for processing these 'robots.txt' files.
NeedsCompilation: yes
URL: https://gitlab.com/hrbrmstr/spiderbar
BugReports: https://gitlab.com/hrbrmstr/spiderbar/issues
License: MIT + file LICENSE
Suggests: 
    covr,
    robotstxt, 
    tinytest
Depends: 
    R (>= 3.2.0)
Encoding: UTF-8
Imports: 
    Rcpp
RoxygenNote: 7.2.3
LinkingTo: Rcpp