Code repository for ValueDCG: Measuring Comprehensive Human Value Understanding Ability of Language Models
You should first add an openai_key_info.py file in the root file, with openai key and url inside.
Then you should set the PYTHONFILE to ValueDCG file.
Finally you can perform the experiments, namely:
git clone https://github.com/zowiezhang/ValueDCG.git
cd ValueDCG
export PYTHONFILE=$(pwd)
pip install -r requirements.txt
python get_response.py
Our LLM parameter settings are as following:
temperture = 0.0,
top_p = 0.95,
seed = 42
And we use get-4o-2024-05-13 version as the evaluator.
Consistency Experiments in experiments
Full experiment record in response