How to compute loss using eval mode in val. py file for YOLOv5

[BIT-QiuYu]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Due to research requirements, I need to calculate the loss function value for each input image in the eval mode of the YOLO V5 model
I modified the run function in the val. py file
The compute_loss variable was specified as ComputeLoss (model) in it

    # Configure
    model.eval()
    compute_loss = ComputeLoss(model)
    cuda = device.type != "cpu"
    is_coco = isinstance(data.get("val"), str) and data["val"].endswith(f"coco{os.sep}val2017.txt")  # COCO dataset
    nc = 1 if single_cls else int(data["nc"])  # number of classes
    iouv = torch.linspace(0.5, 0.95, 10, device=device)  # iou vector for mAP@0.5:0.95
    niou = iouv.numel()

The following error will occur：

Traceback (most recent call last):
  File "val.py", line 626, in <module>
    main(opt)
  File "val.py", line 597, in main
    run(**vars(opt))
  File "xxxxxxxxx/.local/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "val.py", line 299, in run
    compute_loss = ComputeLoss(model)
                   ^^^^^^^^^^^^^^^^^^
  File "utils/loss.py", line 115, in __init__
    h = model.hyp  # hyperparameters
        ^^^^^^^^^
  File "xxxxxxxxxx/.local/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1931, in __getattr__
    raise AttributeError(
AttributeError: 'DetectMultiBackend' object has no attribute 'hyp'

When I imitate the training mode and adding the 'hyp' attribute to the YOLOv5 model using 'data/hyps/hyp.satch-low-yaml' will result in the following error：

Traceback (most recent call last):
  File "val.py", line 626, in <module>
    main(opt)
  File "val.py", line 597, in main
    run(**vars(opt))
  File "xxxxxxx/.local/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "val.py", line 299, in run
    compute_loss = ComputeLoss(model)
                   ^^^^^^^^^^^^^^^^^^
  File "utils/loss.py", line 129, in __init__
    m = de_parallel(model).model[-1]  # Detect() module
        ~~~~~~~~~~~~~~~~~~~~~~~~^^^^
TypeError: 'DetectionModel' object is not subscriptable

I really need the loss value.I look forward to your reply. I would be extremely grateful
If it's not possible to directly modify val. py to achieve the goal, use

torch.hub.load("yolo.pt")

or

from ultralytics import YOLO
Model=YOLO ("yolo5. pt")

and other methods can achieve the goal， I also look forward to your reply. I would greatly appreciate it

Additional

No response

[UltralyticsAssistant]

👋 Hello @BIT-QiuYu, thank you for your interest in YOLOv5 🚀!

For research use cases like yours, it’s great to see modifications aimed at customized loss calculations using val.py. From the error messages provided, it appears that there may be discrepancies in how model attributes and the loss computation are being accessed or initialized.

If this is a 🐛 Bug Report, please provide a minimum reproducible example (MRE), including detailed steps that consistently reproduce the issue. This will help us debug the exact behaviors and assist you better.

If this is a custom training ❓ Question, please ensure you share as much contextual information as possible, including the specific YOLOv5 version being used, the hyperparameters file (if modified), and any environment details such as Python and PyTorch versions. Verifying that you are following the best practices for custom modifications may also help avoid potential pitfalls.

Requirements

Ensure you have the following:

• Python>=3.8.0
• PyTorch>=1.8
• All YOLOv5 repository dependencies installed using pip install -r requirements.txt.

Environments

YOLOv5 supports various environments, such as Jupyter Notebooks with a GPU, cloud environments like Google Cloud and AWS, and running inside Docker containers. Testing across multiple environments may help isolate the issue.

Status

You can also confirm the functionality of your YOLOv5 installation by checking the status of the Continuous Integration (CI) tests, which verify training, validation, inference, export, and benchmarking processes.

This is an automated response to guide you better on your query. An Ultralytics engineer will take a detailed look and assist you further shortly 😊.

[pderrenger]

To compute the loss in evaluation mode using YOLOv5, you can adjust your approach as follows:

1. Issue with model.hyp: The DetectMultiBackend object does not contain the hyp attribute by default in evaluation mode. To resolve this, manually load the hyperparameters (e.g., hyp.scratch-low.yaml) and attach them to the model:

   from utils.general import check_file
   from utils.yaml import yaml_load
   
   model.hyp = yaml_load(check_file('data/hyps/hyp.scratch-low.yaml'))  # Load hyperparameters

2. Issue with model[-1]: The error DetectionModel object is not subscriptable occurs because de_parallel(model).model is not correctly referencing the detection layer. Instead, ensure you pass the correct model structure:

   from utils.general import de_parallel
   compute_loss = ComputeLoss(de_parallel(model))  # Ensure correct model is passed

3. Alternative Approach: Using the Ultralytics YOLO API is simpler and avoids modifying val.py. You can compute losses during validation by iterating over the validation dataset and manually using the ComputeLoss class:

   from ultralytics import YOLO
   from utils.loss import ComputeLoss
   from utils.datasets import create_dataloader
   from utils.general import check_dataset
   
   model = YOLO('yolov5s.pt').model  # Load YOLOv5 model
   model.hyp = yaml_load('data/hyps/hyp.scratch-low.yaml')  # Load hyperparameters
   compute_loss = ComputeLoss(model)
   
   # Load validation data
   data = check_dataset('data/coco128.yaml')  # Update with your dataset
   dataloader = create_dataloader(data['val'], 640, 16, 0, pad=0.5, rect=True, prefix='val: ')[0]
   
   # Compute loss for each batch
   model.eval()
   for batch in dataloader:
       imgs, targets, paths, shapes = batch
       preds = model(imgs)  # Forward pass
       loss, _ = compute_loss(preds, targets)  # Compute loss
       print(f'Loss: {loss}')

This method avoids directly modifying val.py and uses the existing API effectively.

If you encounter further issues, ensure you are using the latest YOLOv5 version from the repository. Let us know if you need additional guidance!