Releases: t1gor/Robots.txt-Parser-Class
Releases · t1gor/Robots.txt-Parser-Class
Version 0.2.5
Bug fixes
- Fix for rules containing
[
and]
. #85
Version 0.2.4
Bug fixes
$parser->setUserAgent($string)
no longer being ignored- Fix for rules containing an end anchor
$
Other improvements
- Improved User-agent parser and group determiner
Version 0.2.3
- PHP 7.1 compatibility
- Rendering function - To render/prettify robots.txt files
- Reformatted and reordered code
Version 0.2.2
- An warning is generated if the
User-agent
format isn't supported, eg when using the full browser-like user-agent string instead of justname/version
. #64 setHttpStatusCode
now returns bool,true
on success orfalse
for any invalid http status code.
Version 0.2.1
- Multiple user-agents in a single group, now share the same rules. #60
Version 0.2
Now also supports:
- Yandex
robots.txt
specifications. - W3C Recommendation
robots.txt
specifications. - The Web Robots Pages
robots.txt
specifications. - The
Clean-Param
directive. - Inline directives.
Features and enhancements:
Clean-Param
array export ability.- Full URLs in
isAllowed
andisDisallowed
now supported.
Ref. Wiki for more information.
Version 0.1
First stable feature complete release!
- Full support for Google's Robots.txt Specifications
Almost every aspect of the official robots.txt specifications has been tested and lots of bugs has been fixed.
Version 0.1 beta
New functions
- Handling HTTP result codes
- Order of precedence for user-agents
Crawl delay and cache delay support
Thanks to our contributors, we can now support the following directives:
- Crawl Delay
- Cache Delay
- Clean Param
Code coverage should also be improved, at least a bit 👍
Great job, guys!
Special thanks to @JanPetterMG !
Current code base released as DEV
v0.0.1-dev codeclimate