Skip to content

Latest commit

 

History

History
154 lines (113 loc) · 7.36 KB

README.md

File metadata and controls

154 lines (113 loc) · 7.36 KB
Raw

Crawlit - Dofus & Dofus-Touch encyclopedia parser - Build your own Dofus API

sosnoob sosnoob

License: CC BY-NC-SA 4.0 Awesome Tweet

In order to respond to a need expressed by the community, I create an encyclopedia parser for Dofus available for :

  • Windows
  • OSX
  • Linux

Download : All versions are available here

⚠️ Because of new parsing method : 429 error limitation removed, You can parse as long as you want

Crawlit was built to be very light, it can run on almost every configurations.

In fact Crawlit is a tool :

  • Developed by the dofus community, for the community
  • 100% performance : Developed to be very light
  • Simple to use | Plug and use
  • Interactive prompt & Command prompt
  • Json file & MongoDB importable file generated
  • Resume parsing feature after errors (connection off / 429 error / ...)
  • English & French encyclopedia supported (french item, english item and soon all other versions)
  • Full Encyclopedia parse available : just execute the npm run dofus-all-fr/npm run dofus-all-en or npm run touch-all-fr/npm run touch-all-en command (be patient 5h & run it in bash shell)

Interactive prompt

App launched App at the end

Command line prompt

You can run the program with those parameter (all lowercase):

Argument Alias Value
-g --game dofus or dofus-touch
-l --language french or english
-c --category categoryName (see list of categories)
-a or -m --all for -a or --maxItem for -m Number for -m/--maxItem (Amount of item you want)
-h --help

Example : npm run crawlit && node ./dist/app.js --game dofus-touch --language english --category monster --maxItem 5

⚠️ If you want all equipments or all weapons : -category argument's value must be written like allequipments and allweapons.

Example : npm run crawlit && node ./dist/app.js --game dofus-touch --language english --category allequipments --all

JSON files also provided in the repository

In case you wan't directly the dofus JSON files, I oftenly upload up-to-date encyclopedia JSON in the data/ folder.

JSON format

An item result example

{  
  "_id":"null : represent your db id",
  "ankamaId": "item id",
  "name":"item name",
  "description":"item description.",
  "lvl":"item lvl",
  "type":"item type",
  "imgUrl":"image url of the item",
  "url":"Item's link",
  "stats":[ "many statistics line", "stat 2", ["..."], "stat n" ],
  "condition":[ "many conditions line", "condition 2", ["..."], "condition n" ],
  "set":{  
     "equipments":[itemId, itemId ...],
     "weapons":[itemId, itemId ...],
  }
}

Getting Started

Clone & access this repository locally :

git clone https://github.com/raczak/crawlit-dofus-encyclopedia-parser.git
cd crawlit-dofus-encyclopedia-parser

1) Install dependencies with npm :

npm install

2) Then at the root folder launch the programm with npm or node :

npm start

⚠️ There is an issue with npm start on some CLI, in this case prefere use npm run build && node lib/app.js.

Supported items

Items state of progress
Equipment 100 %
Weapon 100 %
Set 100 %
Pet 100 %
Mount 100 %
Resource 100 %
Consumable 100 %
Professions 100 %
Recipe 25 %
Classe 100 %
Bestiary 100 %
Harnesse 100 %
Idol 100 %
Sidekick 0 %
Haven Bag 100 %

We also have done an API !

Visite the official non-official Dofus API : DOFAPI

Technologies choice

Crawlit is build with full javascript : NodeJs/Npm, javascript.

Bug & Crash || Proposal ?

For proposal don't hesitate to create a pool request or for a bug/crash an issue. You can also make us aware on bugs channel in discord.

Documentation if you want to contribute (dev)

English Version : Access the wiki to understand the project and contribute to it.

French Version : There is also a french version of the documentation 😃 !

Join the dev Team ? Discord

Contact us on discord (channel general or private message to theukid)

Project sponsors

Contributors


Zakaria RACHEDI
Le Corre Julien
Yannick Milanetto
Darkilen