Improving Transform and Rerank Module

[ritugala]

Description

Major Changes:

1. Added support for using multiple datasets during transformation - this is helpful if a single dataset is too small by itself, or there are too many failing transforms from it. Have changed
2. Changed rerank module to first select dataset and then config. Also to use dataset tags
3. Use Minimum Bayes Risk Decoding (i.e. majority vote) while choosing dataset and config
4. Added task expansion module to better capture requirements of our task
5. Added support for parsing responses from openAI which had Chain of Thought followed by json (search for the rightmost json) - useful in code tasks which may have JSON in the COT also

Minor Changes/Fixes:

1. Removed dataset_index,json and reranking_dataset_index.json file from examples - this seems to have been mistakenly commited
2. Moved make_dataset_from_samples from prompt_based.py -> description_dataset_retriever.py
3. Moved most transform related variables (eg num_votes, max_failed_transforms etc) to be attributes of the DescriptionDatasetRetriever class so that we don't need to pass them around in functions
4. Added exponential backoff / parsing error message from OpenAI to see how long to sleep
5. Loggers would give duplicate messages sometimes, fixed that
6. Updated reranking_dataset_index_tiny.json (used in tests) for using tags
7. Updated tests in tests/dataset_retriever_test.py

Testing

Tested with dummy example + pytest

References

1. I've currently commented out dataset_transformer tests, want to make sure that you guys are happy with the current format before I add tests for it.
2. We should probably add CLI support for all the arguments (and set default values for most ofcourse)

[saum7800]

PR looks good! all important changes. Have left some comments. most minor, some might require little bit of work. Thanks!

[neubig]

Hi, I'm going to trust @saum7800 and @viswavi to take a look at this. It looks exciting, but it's a bit too voluminous for me to take a careful look now.

[viswavi]

Reviewed some of this PR - more to come tomorrow

[viswavi]

I also don't love this - it makes a very specific hardcoded assumption about the rate limit. What if we instead use an on-premise LLM like Llama2 with no rate limit, or we use a version of GPT with a very high rate limit? Then this will probably no longer be required

[viswavi]

On a high-level, these changes look good. I've requested a bunch of small (but important) changes, mostly on minor implementation details, documentation, or code quality things

[viswavi]

Looks good to me!

[saum7800]

Looks good to me too! Great job!