Background
The parsing process has two stages: loading the html and parsing the html.
The api allows the supplied parser to be either a function or an object (map) containing a mixture of cheerio selectors and functions.
The parser may also contain a fetch function to customise the loading of the html page. The fetch function is called during the loading stage.
Problems
- If the parser is a function, it cannot provide a
fetch function
- If the parser is an object, all keys in the object are processed by the parser during the parsing stage; in particular, any
fetch function is called (again) during the parsing stage.
Fixes
- Allow the parser to provide an explicit
parse function.
- When parsing using an object, include only keys that are defined in the api. Other keys, such as
fetch and parse, are ignored.