[Enhancement] Add Chain-of-Thought Prompting + Decoding Controls to inference.py

[AaravJain62677]

Add optional Chain-of-Thought prompting and configurable decoding parameters (temperature, top_p, max_new_tokens) to inference.py. This would make it easier to experiment with reasoning quality for math problems without modifying core code. I’d be happy to work on a PR if this aligns with the project direction.

[Rafiaminhaj]

Hi @AshChadha-iitg 👋

I’m interested in working on this enhancement.

Adding optional Chain-of-Thought prompting along with configurable decoding parameters (temperature, top_p, max_new_tokens) would definitely improve experimentation flexibility and reasoning quality evaluation without requiring core code changes.

My approach would be:

• Introduce optional CLI/config arguments for decoding controls
• Add a toggle for enabling/disabling Chain-of-Thought prompting
• Ensure backward compatibility with existing inference behavior
• Keep implementation modular and clean within inference.py

If this aligns with the project’s direction, could you please assign this issue to me? I’ll ensure the implementation remains well-documented and minimally invasive.

Thank you!