PyTorch Image Data Processing & Classification

Goal of the project

This project implements a full image data pipeline so that a deep learning model can learn efficiently for image classification. It covers data loading, preprocessing, augmentation, train/val/test splitting, and training of a convolutional neural network (CNN) on image data with labels stored in MATLAB format.

---

What’s inside

The work is organized in a single Jupyter notebook, main.ipynb, which includes:

1. Data access

• Datahandler – A custom PyTorch-style dataset that:
  • Loads images from a folder (imagedata/) and labels from imagelabels.mat
  • Supports optional transforms
  • Validates images (corruption check, minimum size)
  • Converts grayscale to RGB when needed
  • Tracks loading errors for debugging

2. Transforms

• Test/inference transforms: resize, center crop, ToTensor, normalization
• Training transforms: same base pipeline plus data augmentation (e.g. random horizontal flip, random rotation, color jitter) to improve generalization

3. Full data setup

• Builds the dataset with the chosen transforms
• Splits data into train, validation, and test sets (e.g. with random_split)
• Optionally builds separate dataset instances per split with the right transform (training vs test)

4. DataLoaders

• DataLoader instances for train, validation, and test with configurable batch size and shuffling
• Basic checks to verify batch counts and data flow

5. Model

• ImageClassificationModel – A convolutional neural network (CNN) for image classification:
  • Convolutional blocks (Conv2d, BatchNorm, ReLU, pooling)
  • Fully connected head for class logits
  • Designed to run on CPU or GPU

6. Training

• Device selection (CUDA if available, else CPU)
• training_function – one-epoch training loop (forward pass, loss, backward, optimizer step)
• Validation loop for monitoring
• Multi-epoch training with loss tracking
• Plotting – training (and optionally validation) loss over epochs with matplotlib

---

Project structure

Pytorch_Image_Data_processing_project/
├── main.ipynb          # Full pipeline: data → transforms → model → training
├── data/               # Data directory
│   ├── imagedata/      # Images (e.g. image_00001.jpg, image_00002.jpg, ...)
│   └── imagelabels.mat # Labels (MATLAB file: 'labels' array, 1-based class indices)
├── README.md
└── requirements.txt   # Python dependencies

Expected data layout:

• Images: data/imagedata/image_00001.jpg, image_00002.jpg, … (5-digit zero-padded index).
• Labels: data/imagelabels.mat with a variable labels (shape (1, n) or (n,)), 1-based class indices (the pipeline converts to 0-based).

---

How to install and run

1. Clone or download the project

cd /path/to/Pytorch_Image_Data_processing_project

2. Create a virtual environment (recommended)

python3 -m venv venv
source venv/bin/activate   # Linux/macOS
# or: venv\Scripts\activate   # Windows

3. Install dependencies

pip install -r requirements.txt

If you use GPU, install PyTorch with CUDA from pytorch.org and then install the rest:

pip install torch torchvision  # choose your CUDA version on the website
pip install pandas scipy Pillow matplotlib

4. Prepare the data

• Place your images in data/imagedata/ with names image_00001.jpg, image_00002.jpg, …
• Place imagelabels.mat in data/ with the labels array (one label per image, 1-based).

5. Open and run the notebook

• Update the dataset path in main.ipynb if needed (e.g. replace /content/drive/MyDrive/... with your local path, e.g. data or ./data).
• Start Jupyter:

jupyter notebook main.ipynb

Or use JupyterLab, VS Code, or Cursor with the Jupyter extension, then run the cells in order.

---

Requirements summary

• Python 3.8+
• PyTorch and torchvision
• pandas, scipy (for reading imagelabels.mat)
• Pillow (PIL) for image loading
• matplotlib for loss plots
• Jupyter to run the notebook

See requirements.txt for version suggestions.