Processing example notebook

README.md

@@ -14,7 +14,7 @@ Then install required python packages
 
 ## Applications
 
-okCupid provides self descriptions, selfies and big questionnaires that are really interesting for anyone interested in psychometrics.
+okCupid provides self descriptions, selfies and big questionnaires that are really useful for anyone interested in psychometrics.
 
 For now I just used it to scrape self descriptions and train an AI to generate new ones. You can check more about it [here](https://mathigatti.com/2021/02/15/okcupid-synthetic-profiles/).
 
@@ -28,16 +28,22 @@ This scraper has two script, the first one downloads the profile data (except th
 
 Using this script and changing your profile details, like gender, sexual orientation and location you can scrape pretty much all users in a given location in okCupid.
 
-You can run it like this
+You can run it like this, users data will be downloaded into _users_ folder
 
 - python users_by_discover.py
 
 You can also try the _users_by_question.py_ script, it search for users that answered specific questions, _questions.csv_ has pretty much all okCupid questions, so I just end up searching for all the possible questions, in the practice users_by_discover.py was more effective into downloading big quantities of users.
 
 ### Download questions
 
+You can run it like this, users answers will be downloaded into _answers_ folder
+
 - python users_by_question.py
 
+### Parsing data
+
+In the testing.ipynb notebook you can check some examples of how to process the data. Users data is downloaded as HTML so I use beautifulSoup to parse it and extract the relevant information. Users questions are in JSON format so it's easier to process.
+
 ## Extra
 
 ### Related datasets