Refactor repo - Clean code - Add YOLOv8 inference

README.md

@@ -67,7 +67,8 @@ food-detection-yolov5
 
 
 <details open> <summary><strong>Dev logs</strong></summary>
-<strong><i>[07/03/2022]</i></strong>  Big refactor. Integrate object detection, image classification, semantic segmentation into one <b><i>Ship of Theseus</i></b>. <br>
+<strong><i>[24/10/2023]</i></strong>  Clean and refactor repo. Integrate YOLOv8 to food detection.<br>
+<strong><i>[07/03/2022]</i></strong>  Big refactor. Integrate object detection, image classification, semantic segmentation into one <b><i>Ship of Theseus</i></b>.<br>
 <strong><i>[31/01/2022]</i></strong>  Update to new YOLOv5 latest versions P5-P6. Can load checkpoints from original repo.<br>
  <strong><i>[26/12/2021]</i></strong> Update app on Android. <br>
  <strong><i>[12/09/2021]</i></strong> Update all features to the web app. <br>
@@ -76,7 +77,7 @@ food-detection-yolov5
 
 
 ## 📔  **Notebook**
-- For inference, use this notebook to run the web app [![Notebook](https://colab.research.google.com/assets/colab-badge.svg)](https://drive.google.com/file/d/1CGEtC65kvoZ-4tcqzeknGrbERvb0beuU/view?usp=sharing)
+- For inference, use this notebook to run the web app [![Notebook](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1X06Y-HSPeHbEWtsXpyal8R1PliiVvpJq?usp=sharing)
 - For training, refer to these notebooks for your own training:
   -  Detection: [![Notebook](https://colab.research.google.com/assets/colab-badge.svg)](https://drive.google.com/file/d/1SywGfyfj3SVrE7VAAl3CshB9s3o8WRXL/view?usp=sharing)
   -  Classification: [![Notebook](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11VzRR8NmJyZGJ-3obkuV0zZAlYAPhCY1?usp=sharing)
@@ -90,10 +91,11 @@ food-detection-yolov5
 
 | Models  | Image Size | Epochs | mAP@0.5 | mAP@0.5:0.95 |
 | ------- | :--------: | :----: | :-----: | :----------: |
-| YOLOv5s |  640x640   |  172   |  90.7   |     67.1     |
-| YOLOv5m |  640x640   |  112   |  89.7   |     66.6     |
-| YOLOv5l |  640x640   |  118   |   94    |      73      |
-| YOLOv5x |  640x640   |   62   |  77.9   |     53.3     |
+| YOLOv5s |  640x640   |  172   |  0.907  |    0.671     |
+| YOLOv5m |  640x640   |  112   |  0.897  |    0.666     |
+| YOLOv5l |  640x640   |  118   |  0.94   |     0.73     |
+| YOLOv5x |  640x640   |   62   |  0.779  |    0.533     |
+| YOLOv8s |  640x640   |   70   |  0.963  |     0.82     |
 
 - Segmentation:
 
@@ -116,19 +118,6 @@ In total, there are 3 implementation versions:
 For those who want to play around with the first version, which remains some features, differ from the new version. You can check out the [v1](https://github.com/lannguyen0910/food-detection-yolov5/tree/v1) branch.
 
  ## 🌟 **Inference**
-- File structure
-```
-this repo
-│   app.py
-└───configs
-│     └───classification          # Contains classification's configurations
-|             └───test.yaml 
-│     └───detection          # Contains detection's configurations
-|             └───....
-│     └───segmentation          # Contains segmentation's configurations
-|             └───....
-
-```
 - Install requirements.
 ```
 pip install -e .
@@ -139,10 +128,14 @@ pip install -e .
 sudo apt-get install ffmpeg
 ``` -->
 
-- Start the app. Safe to run in insecure connection ```http``` on localhost. You can generate SSL certificate to run the app in ```https```.
+- Start the app (Windows). Safe to run in insecure connection ```http``` on localhost. You can generate SSL certificate to run the app in ```https```.
 ```
 run.bat
 ```
+or 
+```python
+python3 app.py
+```
 
 <!-- - Switch between ```CPU``` and ```GPU``` in ```configs```
 ```python
@@ -219,10 +212,10 @@ When a dish is predicted, we provide more information about the nutritional leve
 
 ## 📙 **Credits**
 
-- Custom template: https://github.com/kaylode/theseus.
-- YOLOv5 official repo: https://github.com/ultralytics/yolov5.
-- Semantic segmentation models: https://github.com/qubvel/segmentation_models.pytorch
-- Base code for android app: https://github.com/cmdbug/YOLOv5_NCNN.
-- Ncnn by Tencent: https://github.com/Tencent/ncnn
-- Edamam API: https://developer.edamam.com/food-database-api-docs.
-- Chart.js: https://github.com/chartjs/Chart.js.
+- Custom template: https://github.com/kaylode/theseus
+- YOLOv5: https://github.com/ultralytics/yolov5
+- YOLOv8: https://github.com/ultralytics/ultralytics
+- Timm models: https://github.com/ultralytics/ultralytics
+- Segmentation models: https://github.com/qubvel/segmentation_models.pytorch
+- Edamam API: https://developer.edamam.com/food-database-api-docs
+- Chart.js: https://github.com/chartjs/Chart.js

app.py

@@ -1,19 +1,12 @@
 import os
 import argparse
-import requests
-import cv2
-import numpy as np
-import hashlib
-import time
 
-from PIL import Image
-from flask import Flask, request, render_template, redirect, make_response, jsonify
-from pathlib import Path
-from werkzeug.utils import secure_filename
-from modules import get_prediction
+from flask import Flask
+from flask_cors import CORS
 from flask_ngrok import run_with_ngrok
-from flask_cors import CORS, cross_origin
-from werkzeug.utils import secure_filename
+
+from backend.routes import set_routes
+from backend.constants import UPLOAD_FOLDER, CSV_FOLDER, DETECTION_FOLDER, SEGMENTATION_FOLDER, METADATA_FOLDER
 
 parser = argparse.ArgumentParser('Online Food Recognition')
 parser.add_argument('--ngrok', action='store_true',
@@ -23,248 +16,17 @@
 parser.add_argument('--debug', action='store_true',
                     default=False, help="Run app in debug mode")
 
-ASSETS_DIR = os.path.dirname(os.path.abspath(__file__))
-
 
 app = Flask(__name__, template_folder='templates', static_folder='static')
 CORS(app, resources={r"/api/*": {"origins": "*"}})
 
 app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 1
-
-UPLOAD_FOLDER = './static/assets/uploads/'
-CSV_FOLDER = './static/csv/'
-SEGMENTATION_FOLDER = './static/assets/segmentations/'
-DETECTION_FOLDER = './static/assets/detections/'
-METADATA_FOLDER = './static/metadata/'
-
 app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
 app.config['CSV_FOLDER'] = CSV_FOLDER
 app.config['DETECTION_FOLDER'] = DETECTION_FOLDER
 app.config['SEGMENTATION_FOLDER'] = SEGMENTATION_FOLDER
 
-IMAGE_ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg'}
-VIDEO_ALLOWED_EXTENSIONS = {'mp4', 'avi', '3gpp', '3gp'}
-
-
-def allowed_file_image(filename):
-    return '.' in filename and \
-        filename.rsplit('.', 1)[1].lower() in IMAGE_ALLOWED_EXTENSIONS
-
-
-def allowed_file_video(filename):
-    return '.' in filename and \
-        filename.rsplit('.', 1)[1].lower() in VIDEO_ALLOWED_EXTENSIONS
-
-
-def make_dir(path):
-    if not os.path.exists(path):
-        os.makedirs(path)
-
-
-def file_type(path):
-    filename = path.split('/')[-1]
-    if allowed_file_image(filename):
-        filetype = 'image'
-    elif allowed_file_video(filename):
-        filetype = 'video'
-    else:
-        filetype = 'invalid'
-    return filetype
-
-
-def download(url):
-    """
-    Handle input url from client 
-    """
-
-    make_dir(app.config['UPLOAD_FOLDER'])
-    headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2)',
-               'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
-               'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
-               'Accept-Encoding': 'none',
-               'Accept-Language': 'en-US,en;q=0.8',
-               'Connection': 'keep-alive'}
-    r = requests.get(url, stream=True, headers=headers)
-    print('Image Url')
-
-    # Get cache name by hashing image
-    data = r.content
-    ori_filename = url.split('/')[-1]
-    _, ext = os.path.splitext(ori_filename)
-    filename = hashlib.sha256(data).hexdigest() + f'{ext}'
-
-    path = os.path.join(app.config['UPLOAD_FOLDER'], filename)
-
-    with open(path, "wb") as file:
-        file.write(r.content)
-
-    return filename, path
-
-
-def save_upload(file):
-    """
-    Save uploaded image and video if its format is allowed
-    """
-    filename = secure_filename(file.filename)
-    if allowed_file_image(filename):
-        make_dir(app.config['UPLOAD_FOLDER'])
-        path = os.path.join(app.config['UPLOAD_FOLDER'], filename)
-
-    elif allowed_file_video(filename):
-        make_dir(app.config['VIDEO_FOLDER'])
-        path = os.path.join(app.config['VIDEO_FOLDER'], filename)
-
-    file.save(path)
-
-    return path
-
-
-path = Path(__file__).parent
-
-
-@app.route('/')
-def homepage():
-    resp = make_response(render_template("upload-file.html"))
-    resp.headers['Access-Control-Allow-Origin'] = '*'
-    return resp
-
-
-@app.route('/url')
-def detect_by_url_page():
-    return render_template("input-url.html")
-
-
-@app.route('/webcam')
-def detect_by_webcam_page():
-    return render_template("webcam-capture.html")
-
-
-@app.route('/analyze', methods=['POST', 'GET'])
-@cross_origin(supports_credentials=True)
-def analyze():
-    if request.method == 'POST':
-        out_name = None
-        filepath = None
-        filename = None
-        filetype = None
-        csv_name1 = None
-        csv_name2 = None
-
-        print("File: ", request.files)
-
-        if 'webcam-button' in request.form:
-            # Get webcam capture
-
-            f = request.files['blob-file']
-            ori_file_name = secure_filename(f.filename)
-            filetype = file_type(ori_file_name)
-
-            filename = time.strftime("%Y%m%d-%H%M%S") + '.png'
-            filepath = os.path.join(app.config['UPLOAD_FOLDER'], filename)
-
-            # save file to /static/uploads
-            img = Image.open(f.stream)
-            img.save(filepath)
-
-        elif 'url-button' in request.form:
-            # Get image/video from input url
-
-            url = request.form['url_link']
-            filename, filepath = download(url)
-
-            filetype = file_type(filename)
-
-        elif 'upload-button' in request.form:
-            # Get uploaded file
-
-            f = request.files['file']
-            ori_file_name = secure_filename(f.filename)
-            _, ext = os.path.splitext(ori_file_name)
-
-            filetype = file_type(ori_file_name)
-
-            if filetype == 'image':
-                # Get cache name by hashing image
-                data = f.read()
-                filename = hashlib.sha256(data).hexdigest() + f'{ext}'
-
-                # Save file to /static/uploads
-                filepath = os.path.join(app.config['UPLOAD_FOLDER'], filename)
-
-                np_img = np.fromstring(data, np.uint8)
-                img = cv2.imdecode(np_img, cv2.IMREAD_COLOR)
-                cv2.imwrite(filepath, img)
-
-        # Get all inputs in form
-        iou = request.form.get('threshold-range')
-        confidence = request.form.get('confidence-range')
-        model_types = request.form.get('model-types')
-        enhanced = request.form.get('enhanced')
-        ensemble = request.form.get('ensemble')
-        tta = request.form.get('tta')
-        segmentation = request.form.get('seg')
-
-        ensemble = True if ensemble == 'on' else False
-        tta = True if tta == 'on' else False
-        enhanced = True if enhanced == 'on' else False
-        segmentation = True if segmentation == 'on' else False
-        model_types = str.lower(model_types)
-        min_conf = float(confidence)/100
-        min_iou = float(iou)/100
-
-        if filetype == 'image':
-            # Get filename of detected image
-            out_name = "Image Result"
-            output_path = os.path.join(
-                app.config['DETECTION_FOLDER'], filename) if not segmentation else os.path.join(
-                app.config['SEGMENTATION_FOLDER'], filename)
-
-            output_path, output_type = get_prediction(
-                filepath,
-                output_path,
-                model_name=model_types,
-                tta=tta,
-                ensemble=ensemble,
-                min_conf=min_conf,
-                min_iou=min_iou,
-                enhance_labels=enhanced,
-                segmentation=segmentation)
-
-        else:
-            error_msg = "Invalid input url!!!"
-            return render_template('detect-input-url.html', error_msg=error_msg)
-
-        filename = os.path.basename(output_path)
-        csv_name, _ = os.path.splitext(filename)
-
-        csv_name1 = os.path.join(
-            app.config['CSV_FOLDER'], csv_name + '_info.csv')
-        csv_name2 = os.path.join(
-            app.config['CSV_FOLDER'], csv_name + '_info2.csv')
-
-        if 'url-button' in request.form:
-            return render_template('detect-input-url.html', out_name=out_name, segname=output_path, fname=filename, output_type=output_type, filetype=filetype, csv_name=csv_name1, csv_name2=csv_name2)
-
-        elif 'webcam-button' in request.form:
-            return render_template('detect-webcam-capture.html', out_name=out_name, segname=output_path, fname=filename, output_type=output_type, filetype=filetype, csv_name=csv_name1, csv_name2=csv_name2)
-
-        return render_template('detect-upload-file.html', out_name=out_name, segname=output_path, fname=filename, output_type=output_type, filetype=filetype, csv_name=csv_name1, csv_name2=csv_name2)
-
-    return redirect('/')
-
-
-@app.after_request
-def add_header(response):
-    # Include cookie for every request
-    response.headers.add('Access-Control-Allow-Credentials', True)
-
-    # Prevent the client from caching the response
-    if 'Cache-Control' not in response.headers:
-        response.headers['Cache-Control'] = 'public, no-store, no-cache, must-revalidate, post-check=0, pre-check=0, max-age=0'
-        response.headers['Pragma'] = 'no-cache'
-        response.headers['Expires'] = '-1'
-    return response
-
+set_routes(app)
 
 if __name__ == '__main__':
     if not os.path.exists(UPLOAD_FOLDER):
@@ -292,4 +54,4 @@ def add_header(response):
         host = hostname[0]
 
         app.run(host=host, port=port, debug=args.debug, use_reloader=False,
-                ssl_context='adhoc')
+                ssl_context='adhoc')

backend/constants.py

@@ -0,0 +1,11 @@
+UPLOAD_FOLDER = './static/assets/uploads/'
+VIDEO_FOLDER = './static/assets/videos/'
+CSV_FOLDER = './static/csv/'
+SEGMENTATION_FOLDER = './static/assets/segmentations/'
+DETECTION_FOLDER = './static/assets/detections/'
+METADATA_FOLDER = './static/metadata/'
+
+IMAGE_ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg'}
+VIDEO_ALLOWED_EXTENSIONS = {'mp4', 'avi', '3gpp', '3gp'}
+
+CACHE_DIR = './weights'

analyzer/api.py → backend/edamam/api.py

@@ -3,7 +3,7 @@
 from tqdm import tqdm
 from .secret import API, get_response
 
-DATABASE = "./analyzer/db.json"
+DATABASE = "./backend/edamam/db.json"
 
 
 def make_request(api_name, params, headers):