GRiD: Dependency Matters - Enhancing LLM Reasoning with Explicit Knowledge Grounding

This repository contains the code for the paper "Dependency Matters: Enhancing LLM Reasoning with Explicit Knowledge Grounding" (NeurIPS 2025).

Overview

This project implements and evaluates various reasoning strategies for large language models (LLMs) on multiple question-answering benchmarks. The codebase supports training and evaluation of different reasoning methods including ReAct, Chain-of-Thought (CoT), direct reasoning, and several other approaches.

Supported Methods

• ReAct: Reasoning and Acting
• CoT: Chain-of-Thought
• Direct: Direct answer generation
• Think: Thinking-based reasoning
• Think-Reason: Combined thinking and reasoning
• Disentangle: Disentangled reasoning approach
• Reasoner: Dedicated reasoner models

Supported Datasets

• TruthfulQA: Truthful question answering
• StrategyQA: Strategic reasoning questions
• CommonsenseQA: Commonsense question answering
• GPQA: Graduate-level question answering

Quick Start

Environment Setup

Install required dependencies:

pip install transformers datasets unsloth peft accelerate deepspeed wandb

Training

Train a model using one of the provided scripts. For example, to train a ReAct model on TruthfulQA:

bash train_react_TruthfulQA_template.sh

Other training scripts follow the pattern: train_{method}_{dataset}_template.sh

Evaluation

Evaluate a trained model using the evaluation scripts:

bash eval_react_TruthfulQA_template.sh

Evaluation scripts follow the pattern: eval_{method}_{dataset}_template.sh

Project Structure

GRiD/
├── dataset/              # Dataset files
├── ckpts/                # Model checkpoints
├── eval_results/         # Evaluation results
├── deepspeed_configs/    # DeepSpeed configurations
├── train_*.py           # Training scripts (Python)
├── train_*.sh           # Training scripts (Shell)
├── eval_*.py            # Evaluation scripts (Python)
├── eval_*.sh            # Evaluation scripts (Shell)
├── utils.py             # Utility functions
└── README.md            # This file

Key Features

• Multiple Reasoning Methods: Compare different reasoning strategies
• Multiple Datasets: Evaluate on diverse QA benchmarks
• Efficient Training: Supports LoRA, quantization, and DeepSpeed
• Flexible Configuration: Easy to modify training and evaluation parameters

Citation

If you use this code in your research, please cite:

@inproceedings{grid2025,
  title={Dependency Matters: Enhancing LLM Reasoning with Explicit Knowledge Grounding},
  author={Xiangyu Wen, Min Li, Junhua Huang, Zhijian Xu, Zeju Li, Yongxiang Huang, Mingxuan Yuan, Qiang Xu},
  booktitle={Advances in Neural Information Processing Systems},
  year={2025}
}