v1.0.0

Bug Fixes

• fix freeze when exception while pageEvaluate (collect page data), add html_size (5b2541e)
• h1 trim (67fbc0d)
• ignore /?display= (2b08a55)
• ignore bitrix redirects (b310334)
• ignore case on extension check (be5407e)
• move basic options to index.js (f7e1d6b)
• move canonical after url (3c74a10)
• remove url duplicates (110b379)
• save csv for Excel (c9e3004)

Features

• add referer page (572411f)
• add skip_static to options (7981e9b)
• fields presets (0577b39)
• ignore css and js (a203275)
• is_canonical (e6b0377)
• move sites list to data/sites.conf (68f16ca)
• optional encode csv to win1251 (f025b7f)
• progress, show page count, time per page (b6aa6f4)
• progress: more verbose output, queue size, events (a115e7c)
• request_time, h1, h1-h4 count, dom_count (29c2f86)
• scrap site list (de18f1e)