diff --git a/CODE-OF-CONDUCT.md b/CODE-OF-CONDUCT.md new file mode 100644 index 0000000..9dc8b5c --- /dev/null +++ b/CODE-OF-CONDUCT.md @@ -0,0 +1,85 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity +and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our community include: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience +- Focusing on what is best not just for us as individuals, but for the overall community + +Examples of unacceptable behavior include: + +- The use of sexualized language or imagery, and sexual attention or advances of any kind +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or email address, without their explicit permission +- Other conduct which could reasonably be considered inappropriate in a professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting +via an official social media account, or acting as an appointed representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement. All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of actions. + +**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.0, available at +https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. + +Community Impact Guidelines were inspired by +[Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity). + +[homepage]: https://www.contributor-covenant.org + +For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. +Translations are available at https://www.contributor-covenant.org/translations. \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bd48e2f..bdc3e4f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,163 +1,272 @@ -# Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss -## Contributing To Our Projects +# Contributing to Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss projects -  +Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. -The Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. +## Ground rules & expectations -The following is a guide on how to contribute to Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss Github projects and **_should be followed in all cases_**. +Before we get started, here are a few things we expect from you (and that you should expect from others): -  +- Be kind and thoughtful in your conversations around this project. We all come from different backgrounds and projects, which means we likely have different perspectives on "how open source is done." Try to listen to others rather than convince them that your way is correct. +- This project is released with a [Contributor Code of Conduct](CODE-OF-CONDUCT.md). By participating in this project, you agree to abide by its terms. +- Please ensure that your contribution complies with this document. If it does not, you will need to address and fix all issues before we can merge your contribution. +- When adding content, please consider if it is widely valuable. -# Table Of Contents - -- [Repository Issues](#repository-issues) - - [Bug reports](#bug-reports) - - [Feature requests](#feature-requests) - - [Feature proposals](#feature-proposals) -- [Creating a fork of the repository](#creating-a-fork-of-the-repository) -- [Repository structure](#repository-structure) -- [Documentation](#documentation) -- [Installation Scripts](#documentation) -- [Naming](#documentation) -- [Attribution](#documentation) -- [.md Files](#documentation) -- [Push changes to forked repository](#documentation) -- [Pull requests](#documentation) -- [Pull requests process](#documentation) -- [Code of conduct](#documentation) - - [Our pledge](#documentation) - - [Our standards](#documentation) - - [Our responsibilities](#documentation) +## Overview +Being an Open Source project, everyone can contribute, provided that you respect the following points: +- Before contributing any code, the author must make sure all the tests work (see below how to launch the tests). +- Developed code must adhere to the syntax guidelines enforced by the linters. +- Code must be developed following the [SemVer (Semantic Versioning 2.0.0)](https://semver.org/) branching model. +- For any new feature added, unit tests must be provided, following the example of the ones already created. -  +## How to contribute + +If you'd like to contribute, start by searching through the issues and pull requests to see whether someone else has raised a similar idea or question. -# Repository Issues +If you don't see your idea listed, and you think it fits into the goals of this guide, do one of the following: + +- Bug Report +- Feature Proposal +- Feature Request +- Vulnerability Report + +### Repository Issues The first step is to head to our repository issues tab and decide how you would like to contribute. -![Repository Issues](assets/images/repo-issues.png) +![Repository Issues](assets/images/repo-issues.jpg) -## Bug reports +### Bug reports -![Bug Reports](assets/images/bug-report.png) +![Bug Reports](assets/images/bug-report.jpg) If you would like to contribute bug fixes or make the team aware of bugs you have identified in the project, please raise a **Bug report** issue in the [issues section](issues/new/choose) section. A template is provided that will allow you to provide your suggestions for your bug report / bug fix(es) which will be reviewed by the team. Bug fix issues are the first step to creating a pull request for bug fixes, once you have created your issue and it has been approved you can proceed with your bug fixes. -## Feature requests +### Feature proposals -![Bug Reports](assets/images/feature-request.png) +![Feature Proposals](assets/images/feature-proposals.jpg) + +If you would like to contribute new features to the project, please raise a **Feature proposal** issue in the [issues section](issues/new/choose) section. A template is provided that will allow you to provide your suggestions for your feature proposal. + +Feature proposal issues are the first step to creating a pull request for feature proposals, once you have created your issue and it has been approved you can proceed with your feature proposal. + +### Feature requests + +![Feature requests](assets/images/feature-request.jpg) If you would like to suggest a new feature/new features for this project, please raise a **Feature request** issue in the [issues section](issues/new/choose) section. A template is provided that will allow you to provide your suggestions for your feature request. -## Feature proposals +### Community -![Bug Reports](assets/images/feature-proposals.png) +Discussions about the Open Source Guides take place on this repository's [Issues](issues) and [Pull Requests](pulls) sections, or the [discussions](discussions). Anybody is welcome to join these conversations. -If you would like to contribute new features to the project, please raise a **Feature proposal** issue in the [issues section](issues/new/choose) section. A template is provided that will allow you to provide your suggestions for your feature proposal. +Wherever possible, do not take these conversations to private channels, including contacting the maintainers directly. Keeping communication public means everybody can benefit and learn from the conversation. -  +### Getting Started -# Creating a fork of the repository +In order to start contributing: -![Creating a fork of the repository](assets/images/fork.png) -If you are going to be contributing code or documentation to the repository you will need to make a fork of the current development branch. The current development branch will always be the highest branch or master. Once you have created a fork of the repository, you will be able to clone the repository to a location on your development machine using terminal and Git or Github Desktop if using Windows. +![Fork](assets/images/fork.jpg) -  +1. Fork this repository clicking on the "Fork" button on the upper-right area of the page. + +2. Clone your just forked repository: -# Repository structure +```bash +git clone https://github.com/YourAccount/hias-all-oneapi-classifier.git +``` + +3. Add the main HIAS MQTT IoT Agent repository as a remote to your forked repository: + +```bash +git remote add origin https://github.com/AIIAL/hias-all-oneapi-classifier.git +``` + +Before starting your contribution, remember to synchronize the latest `develop` branch in your forked repository with the `develop` branch in the main repository. To do this, use following these steps + +1. Change to your local `develop` branch (in case you are not in it already): + +```bash +git checkout develop +``` + +2. Fetch the remote changes: + +```bash +git fetch origin +``` + +3. Merge them: + +```bash +git rebase origin/1.0.0 +``` + +Contributions following these guidelines will be added to the `develop` branch, and released in the next version. The release process is explained in the _Releasing_ section below. + +### Documentation + +Changes you make to the code in the repository or new projects that you make should be supported with documentation added to the **docs** directory. + +It is the contributor's responsibility to ensure that the documentation is up to date. If you are contributing to an existing repository you will ensure that these documents are updated and/or added to to reflect your changes. + +We use [MKDocs](https://www.mkdocs.org/) along with [Read the Docs](https://docs.readthedocs.io/en/stable/index.html). Use the [Getting Started with MkDocs](https://docs.readthedocs.io/en/stable/intro/getting-started-with-mkdocs.html) guide to find out how to update/create documentation for the project. + +To start the MKDocs server on use the following command: + +``` bash +mkdocs serve +``` +To start the MKDocs on a specific IP and port use the following command, updating your IP and port as required: + +``` bash +mkdocs serve -a 192.168.1.578:8000 +``` +Remember to update the [mkdocs.yml](mkdocs.yml) file to match your documentation structure. + +### Repository structure + +Repository structures for HIAS AI Agents **must be followed exactly** for all contributions. Pull Requests that do not follow this structure will be rejected and closed with no further discussion. ``` - Project Root (Directory) - - config (Directory) * - - config.json (File) * - - credentials.json (File) * - - modules (Directory) * - - AbstractAgent.py (File) * - - AbstractData.py (File) * - - AbstractModel.py (File) * - - AbstractServer.py (File) * - - helpers.py (File) * - - data.py (File) * - - model.py (File) * - - mqtt.py (File) * - - server.py (File) * - - documentation (Directory) + - assets (Directory) + - images (Directory) + - project-banner.jpg (Image) + - bug-report.jpg (Image) + - feature-proposal.jpg (Image) + - feature-request.jpg (Image) + - fork.jpg (Image) + - repo-issues.jpg (Image) + - configuration (Directory) + - config.json (File) + - docs (Directory) + - img (Directory) + - project-banner.jpg (image) - installation (Directory) - - Ubuntu - - Windows - - Raspberry Pi OS + - ubuntu.md (File) + - usage (Directory) + - ubuntu.md (File) + - index.md (File) - logs (Directory) - - logfiles - - assets - - media - - images (Directory) - - videos (Directory) - - model - - data (Directory) * - - test (Directory) * - - train (Directory) * - - plots (Directory) * + - Auto generated log files + - modules (Directory) + - AbstractClassifier.py (File) + - AbstractData.py (File) + - AbstractModel.py (File) + - AbstractServer.py (File) + - helpers.py (File) + - augmentation.py (File) + - data.py (File) + - model.py (File) + - server.py (File) + - model (Directory) + - data (Directory) + - test (Directory) + - train (Directory) + - plots (Directory) - model.json (File) - weights.h (File) + - notebooks (Directory) + - classifier.ipynb (File) - scripts - install.sh (File) - agent.py (File) - - Project README (File) + - CODE-OF-CONDUCT.md (File) + - CONTRIBUTING.md (File) + - LICENSE (File) + - mkdocs.yml (File) + - README.md (File) ``` -  +**Directories and files may be added to the above structure as required, but none must be removed.** -# Documentation +### Installation Scripts -All projects should have documentation, changes you make to the code in the repository or new projects that you make should also have documentation. In the structures above you can see the **Documentation** directories, and the **Installation** file. This installation file should contain a step by step guide to how to install your project, while directions of use will be a high level tutorial in your project README. If you are contributing an existing repository you will ensure that these documents are updated to reflect your changes. +The default installation script is [install.sh](scripts/install.sh) found in the [scripts](scripts) directory. -  +You must include the installation commands for all libraries for the project using apt/pip/make etc. Replace **# DEVELOPER TO ADD INSTALLATION COMMANDS FOR ALL REQUIRED LIBRARIES (apt/pip etc)** with the relevant installation commands. If you are contributing an existing repository you will ensure that these scripts are updated to reflect your changes. -# Installation Scripts +### Configuration -You will notice the **Scripts** directory above, this is home to any scripts that make it easier for the end use to set up and / or use your project. By default every project must have an installation script when installations are required. If you are contributing an existing repository you will ensure that these scripts are updated to reflect your changes. +The project configuration file [config.json](configuration/config.json) can be found in the [configuration](configuration) directory. -  +All configurable variables should be held within this file and used wherever relevant throughout the project. -# Naming +The [helpers file](modules/helpers.py) loads the configuration and makes it available as `helpers.confs`. -We use CamelCase for our directory and file names (except images), we would like this to stay uniform across all of our projects. Please use descriptive but short names where possible, and make sure you do not use spaces in directory and file names. +You may remove redundent objects/arrays/values. from the configuration and/or add new ones. -  +### Project Images -# Attribution +Images used in the project must be **jpg**. You must own rights to images you upload to the project, or include attribution. Contributors are solely responsible for any images they publish to our Github. -Whenever relevant you should add yourself to the contributors section of the READMEs. If you have a public profile on the Peter Moss Leukemia AI Research website, you should use this in your link. Please see the contributing section of any of our READMEs for examples. +### Naming Scheme -  +The following naming scheme must be used: -# .md Files +- **Directories:** Snake case (snake_case) +- **Abstract Files:** CamelCase (CamelCase) +- **Files:** Spinal case (spinal-case) +- **Images:** Spinal case (spinal-case) -Each .md file in the repository should use the same header. An example of this is as follows: +Please use descriptive but short names, and make sure to not use spaces in directory and file names. -``` -# Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss -## Acute Lymphoblastic Leukemia oneAPI Classifier +### Headers -![Acute Lymphoblastic Leukemia oneAPI Classifier](assets/images/all-oneapi-classifier-2020.png) -``` -If the .md file is the README in the project root, it should including the following markdown, ensuring the repository URLS and versions are correct. +All Python files must include the following header, replacing **Module Title** with a short but descriptive title for the module, and **Module Description** with a paragraph explaining what the module is for. ``` -[![CURRENT RELEASE](https://img.shields.io/badge/CURRENT%20RELEASE-0.0.0-blue.svg)](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/tree/0.0.0) [![UPCOMING RELEASE](https://img.shields.io/badge/CURRENT%20DEV%20BRANCH-0.1.0-blue.svg)](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/tree/0.1.0) [![Contributions Welcome!](https://img.shields.io/badge/Contributions-Welcome-lightgrey.svg)](CONTRIBUTING.md) [![Issues](https://img.shields.io/badge/Issues-Welcome-lightgrey.svg)](issues) [![LICENSE](https://img.shields.io/badge/LICENSE-MIT-blue.svg)](LICENSE) -``` +#!/usr/bin/env python3 +""" Module Title. + +Module Description. + +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files(the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and / or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +Contributors: + +""" ``` +#### Attribution + +- When you create a new module you should add your name to the **Contributors** section. +- When you make a change to an existing module you should add your name to the **Contributors** section below existing contributors. You must not remove existing contributors. + +### Footers + +All READMEs and documentation should include the following footer: + +``` # Contributing -The Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. +Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. -Please read the [CONTRIBUTING](CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page. +Please read the [CONTRIBUTING](CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find our code of conduct in the [Code of Conduct](CODE-OF-CONDUCT.md) document. ## Contributors - [Adam Milton-Barker](https://www.leukemiaairesearch.com/association/volunteers/adam-milton-barker "Adam Milton-Barker") - [Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain @@ -165,7 +274,7 @@ Please read the [CONTRIBUTING](CONTRIBUTING.md "CONTRIBUTING") document for a fu   # Versioning -We use SemVer for versioning. +We use [SemVer](https://semver.org/) for versioning.   @@ -178,110 +287,192 @@ This project is licensed under the **MIT License** - see the [LICENSE](LICENSE " We use the [repo issues](issues "repo issues") to track bugs and general requests related to using this project. See [CONTRIBUTING](CONTRIBUTING.md "CONTRIBUTING") for more info on how to submit bugs, feature requests and proposals. ``` -The project README contributors should contain a list of all contributors in the entire repository. +Remember to use **relative URLs**, and in the case of footers in the [docs](docs) folder, you must us **absolute URLs**. -  +The contributors section should include a list of contributors that have contributed to the related document. In the case of the README footer, the Contributors section should include a list of contributors that have contributed to **any** part of the project. -# Push changes to forked repository +You should add your details below existing contributors. Details should include: -When you have made your changes, you need to push them to your forked repository. This process requires a commit with a message about what your commit is, and then finally a push to the repository which will place your changes or created files to your forked repository. +- Name +- Company/University etc +- Position +- City +- Country -Show the status of all changed/added files. +Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss Volunteers should use the following: -``` -git status -``` +- Name +- Association name +- Association Position +- City +- Country -You may want to do check the differences between changed files, you can do this using the following command. +### Branching model -``` -git diff -``` +There are one special branch in the repository: -Add all changes ready to commit. (You may not want to add all changed files, in this case please follow instructions to add single files to commit that you were shown using git status). To repeat, the following will add all changes shown by using git status to your commit. +- `main`: contains the tagged and released version +- `develop`: is the `dev` branch and contains the latest development code. New features and bugfixes are always merged to `dev` branch. -``` -git add . -``` +In order to start developing a new feature or refactoring, a new branch will be created following the SemVer scheme: -Commit your changes. +Given a version number MAJOR.MINOR.PATCH, increment the: -``` -git commit -m "Your descriptive commit message" -``` +- `MAJOR` version when you make incompatible code changes, +- `MINOR` version when you add functionality in a backwards compatible manner, and +- `PATCH` version when you make backwards compatible bug fixes. -Push changes to your fork. +- If `MAJOR` is 0, then Minor could mean the version is not backwards compatible +- If `MINOR` is 1, this means the release is stable. -``` -git push -``` +The branch will be created by our team depending on the nature of your issue. Once the new functionality has been completed, a Pull Request will be created from the feature branch to the relevant branch. Remember to check both the linters, and +the tests before creating the Pull Request. -You can also Github Desktop if on Windows as shown in the image above. +In order to contribute to the repository, the same scheme should be replicated in the forked repositories, so the new features or fixes should all come from the current version of `dev` branch and end up in the current `dev` branch again. -  +### PEP 8 -- Style Guide for Python Code -# Pull requests +All Python projects must align with the [PEP 8 -- Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/). -When you are satisfied with your changes you can submit a pull request that will request that your changes be pulled into the latest development branch. +### CII Best Practices -## Pull Request Process +All projects must align with the [CII Best Practice](https://www.construction-institute.org/resources/knowledgebase/best-practices). -1. Ensure all documentation has been updated and matches the style of our repository layouts, including repository directory structures etc. -2. Add new README.md files for new directories / ** init **.py files for empty directories. -3. Ensure that your READMEs, documentation and code match the format/design of the rest of the repoistory. -4. Include the project header banner on all documentation pages / READMEs. -5. Incude the Contributing, Versioning, Licensing, Bugs/Issues information in all READMEs and documentation. -6. Do not ever upload licensed software or images that you do not own the rights to. -7. Submit your Pull Request with an accurate description of the changes you have made. -8. Your Pull Request will be reviewed by the team. +### Changelog -  +The project contains a changelog that is automatically created from the description of the Pull Requests that have been merged into develop, thanks to the [Release Drafter GitHub action](https://github.com/marketplace/actions/release-drafter). -# Code of conduct +### Releasing -Please note we have a code of conduct, please follow it in all your interactions with the project. +The process of making a release simply consists of creating the release in Github and providing the new tag name, this task is carried out by our team. -## Our Pledge +### Version numbers -In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. +The version number will change for each release, following the SemVer scheme described previously. -## Our Standards +### Bugfix in releases -Examples of behavior that contributes to creating a positive environment include: +When a bug is found affecting a release, a branch will be created from the `main` branch. As a part of the patch, the release version will be increased in it's last number (Z). The patch then will be merged (via pull request (PR)) to the `main` branch, and a new version will be released. -- Using welcoming and inclusive language -- Being respectful of differing viewpoints and experiences -- Gracefully accepting constructive criticism -- Focusing on what is best for the community -- Showing empathy towards other community members +### Commits -Examples of unacceptable behavior by participants include: +Commits should be [atomic](https://en.wikipedia.org/wiki/Atomic_commit), make commits to your fork until you have resolved/completed the work specified in your issue before submitting your PR, this keeps an easy to follow history of what work has been carried out and makes the review process easier and quicker -- The use of sexualized language or imagery and unwelcome sexual attention or advances -- Trolling, insulting/derogatory comments, and personal or political attacks -- Public or private harassment -- Publishing others' private information, such as a physical or electronic address, without explicit permission -- Other conduct which could reasonably be considered inappropriate in a professional setting +When making a commit, the subjects of each commit should be as follows, where XXX represents the issue number: -## Our Responsibilities +#### Commit Title -Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. +##### Bug Fixes -Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or -permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. +- fix #xxx Issue Name +- fixes #xxx Issue Name +- fixed #xxx Issue Name -## Scope +##### Partial Resolutions -This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. +- partially resolves #xxx Issue Name +- partially resolved #xxx Issue Name -## Enforcement +##### Resolution -Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at github@leukemiaresearchassociation.ai All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. +- resolves #xxx Issue Name +- resolved #xxx Issue Name -Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. +##### Alignment -  +- aligns with #xxx Issue Name + +##### Closure + +- close #xxx Issue Name +- closes #xxx Issue Name +- closed #xxx Issue Name + +#### Commit Description -# Attribution +Your commit description should include a detailed description of the changes that have been made. + +#### Committing + +When you are ready to commit, you should do the following: + +##### Show The Status Of All Changed/Added/Deleted files + +``` +git status +``` + +##### Diff + +You may want to check the differences between changed files, you can do this using the following command. + +``` +git diff +``` + +##### Add All Changes + +The following will add all changes shown by git status to your commit. + +``` +git add . +``` + +##### Add One Or More Changes + +``` +git add file1 file2 file5 +``` + +##### Commit Added Changes + +Commit your added changes to your local repository, remember to follow the [Commit Title](#commit-title) & [Commit Description](#commit-description) guides above. + +To create your commit with both a title and description, use the following command which states the commit fixes issue ID 1 and provides a detailed description: + +``` +git commit -m "fixes #1" -m "Fixes the documentation typos described in issue #1" +``` + +### Push Your Changes + +When you have made your changes, ensured you have aligned with the procedures in this document, and made your commits to your local repository aligning with the guide above, you need to push your changes to your forked repository. + +Push changes to your fork by using the following command: + +``` +git push +``` -This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org/version/1/4), version 1.4. \ No newline at end of file +### Pull Request protocol + +Contributions to the HIAS MQTT IoT Agent repository are done using a PR. The detailed "protocol" used in such PR is described below: + +* Direct commits to main or develop branches (even single-line modifications) are not allowed. Every modification has to come as a PR to the latest `dev branch` +* PRs implement/fix submitted issues, the issue number has to be referenced in the subject of the relevant commit and PR +* Anybody is welcome to provide comments to the PR (either direct comments or using the review feature offered by Github) +* Use *code line comments* instead of *general comments*, for traceability reasons (see comments lifecycle below) +* Comments lifecycle + * Comment is created, initiating a *comment thread* + * New comments can be added as responses to the original one, starting a discussion + * After discussion, the comment thread ends in one of the following ways: + * `Fixed in ` in case the discussion involves a fix in the PR branch (which commit hash is included as reference) + * `NTC`, if finally nothing needs to be done (NTC = Nothing To Change) +* PR can be merged when the following conditions are met: + * All comment threads are closed + * All the participants in the discussion have provided a `LGTM` general comment (LGTM = Looks good to me) + * All documentation has been updated to reflect your changes. + * No proprietory software or images have been added. + +* Self-merging is not allowed (except in rare and justified circumstances) + +Some additional remarks to take into account when contributing with new PRs: + +* PR must include not only code contributions, but their corresponding pieces of documentation (new or modifications to existing one) and tests +* Documentation must be added to the **docs** folder +* In the case empty directories need to be uploaded, add a `.gitkeep` file inside. +* The project banner is included in all documentation +* Contributing, Versioning, Licensing, Bugs/Issues footer is included in all information +* Contributors have been added to all Contributors footers +* PR modifications must pass full regression based on existing tests in addition to whichever new test added due to the new functionality +* PR should be of an appropriated size that makes review achievable. Too large PRs could be closed with a "please, redo the work in smaller pieces" without any further discussion \ No newline at end of file diff --git a/README.md b/README.md index ccef212..c2bcab7 100644 --- a/README.md +++ b/README.md @@ -1,94 +1,55 @@ # Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss ## Acute Lymphoblastic Leukemia oneAPI Classifier -![Acute Lymphoblastic Leukemia oneAPI Classifier](assets/images/all-oneapi-classifier-2020.png) +![Acute Lymphoblastic Leukemia oneAPI Classifier](assets/images/project-banner.jpg) -[![CURRENT RELEASE](https://img.shields.io/badge/CURRENT%20RELEASE-1.0.0-blue.svg)](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/tree/1.0.0) [![UPCOMING RELEASE](https://img.shields.io/badge/CURRENT%20DEV%20BRANCH-1.1.0-blue.svg)](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/tree/1.1.0) [![Contributions Welcome!](https://img.shields.io/badge/Contributions-Welcome-lightgrey.svg)](CONTRIBUTING.md) [![Issues](https://img.shields.io/badge/Issues-Welcome-lightgrey.svg)](issues) [![LICENSE](https://img.shields.io/badge/LICENSE-MIT-blue.svg)](LICENSE) +[![CURRENT RELEASE](https://img.shields.io/badge/CURRENT%20RELEASE-1.1.0-blue.svg)](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/tree/release-1.1.0) [![UPCOMING RELEASE](https://img.shields.io/badge/DEV%20BRANCH-Develop-blue.svg)](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/tree/develop) [![Contributions Welcome!](https://img.shields.io/badge/Contributions-Welcome-lightgrey.svg)](CONTRIBUTING.md) [![Issues](https://img.shields.io/badge/Issues-Welcome-lightgrey.svg)](issues) -  +[![PEP8](https://img.shields.io/badge/code%20style-pep8-orange.svg)](https://www.python.org/dev/peps/pep-0008/) [![Documentation Status](https://readthedocs.org/projects/hias-acute-lymphoblastic-leukemia-oneapi-classifier/badge/?version=latest)](https://hias-acute-lymphoblastic-leukemia-oneapi-classifier.readthedocs.io/en/latest/?badge=latest) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5226/badge)](https://bestpractices.coreinfrastructure.org/projects/5226) -# Table Of Contents - -- [Introduction](#introduction) -- [DISCLAIMER](#disclaimer) -- [Motivation](#motivation) -- [Acute Lymphoblastic Leukemia](#acute-lymphoblastic-leukemia) - - [ALL-IDB](#all-idb) - - [ALL_IDB1](#all_idb1) -- [Acute Lymphoblastic Leukemia Tensorflow Classifier 2020](#acute-lymphoblastic-leukemia-tensorflow-classifier-2020) -- [Intel Technologies](#intel-technologies) - - [Intel® oneAPI Toolkits (Beta)](#intel-oneapi-toolkits-beta) - - [Intel® Distribution for Python](#intel-distribution-for-python) - - [Intel® Optimization for TensorFlow](#intel-optimization-for-tensorflow) - - [Intel® Distribution of OpenVINO™ Toolkit](#intel-distribution-of-openvino-toolkit) - - [Intel® Movidius™ Neural Compute Stick 2](#intel-movidius-neural-compute-stick-2) -- [Acute Lymphoblastic Leukemia oneAPI Classifier 2021](#acute-lymphoblastic-leukemia-oneapi-classifier-2021) -- [GETTING STARTED](#getting-started) -- [Contributing](#contributing) - - [Contributors](#contributors) -- [Versioning](#versioning) -- [License](#license) -- [Bugs/Issues](#bugs-issues) +[![LICENSE](https://img.shields.io/badge/LICENSE-MIT-blue.svg)](LICENSE) ![SemVer](https://img.shields.io/badge/semver-2.0.0-blue)   # Introduction -The **Acute Lymphoblastic Leukemia (ALL) oneAPI Classifier** is an open-source classifier programmed using the [Intel® Distribution for Python*](https://software.intel.com/content/www/us/en/develop/tools/distribution-for-python.html) and trained using [Intel® Optimization for TensorFlow*](https://software.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation-guide.html) (Tensorflow 2.1). The model is deployed on a Raspberry 4 using [Intel® Distribution of OpenVINO™ Toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html) and inference is carried out using the Intel Movidius Neural Compute Stick 2. +The **Acute Lymphoblastic Leukemia (ALL) oneAPI Classifier** is an open-source classifier programmed using the [Intel® Distribution for Python*](https://software.intel.com/content/www/us/en/develop/tools/distribution-for-python.html) and trained using [Intel® Optimization for TensorFlow*](https://software.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation-guide.html). The model is deployed on a Raspberry 4 using [Intel® Distribution of OpenVINO™ Toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html) and inference is carried out using the [Intel® Neural Compute Stick 2 (Intel® NCS2)](https://software.intel.com/content/www/us/en/develop/hardware/neural-compute-stick.html).   # DISCLAIMER -This project should be used for research purposes only. The purpose of the project is to show the potential of Artificial Intelligence for medical support systems such as diagnosis systems. +_This project should be used for research purposes only. The purpose of the project is to show the potential of Artificial Intelligence for medical support systems such as diagnostic systems._ -Although the classifier is accurate and shows good results both on paper and in real world testing, it is not meant to be an alternative to professional medical diagnosis. +_Although the model is accurate and shows good results both on paper and in real world testing, it is trained on a small amount of data and needs to be trained on larger datasets to really evaluate it's accuracy._ -Developers that have contributed to this repository have experience in using Artificial Intelligence for detecting certain types of cancer. They are not doctors, medical or cancer experts. +_Developers that have contributed to this repository have experience in using Artificial Intelligence for detecting certain types of cancer. They are not doctors, medical or cancer experts._   # Motivation -The motivation for this project came from the interest in exploring how Intel technologies could be used to create an improved version of the [Acute Lymphoblastic Leukemia Tensorflow 2020](#all-tensorflow-2020) project. - -The goal is to create an improved computer vision model that was capable of detecting Acute Lymphoblastic Leukemia in unseen images of periphial blood samples with high accuracy and efficiency, running a low powered/low resources Raspberry Pi 4. +The motivation for this project came from the interest in exploring how Intel technologies could be used to create an improved version of the [Acute Lymphoblastic Leukemia Tensorflow 2020](https://github.com/AMLResearchProject/ALL-Tensorflow-Classifier-2020) project.   # Acute Lymphoblastic Leukemia -Acute Lymphoblastic Leukemia (ALL), also known as Acute Lymphocytic Leukemia, is a cancer that affects the Lymphoid blood cell lineage. Unlike AML, ALL only affects the white blood cells, namely, Lymphocytes. Lymphocytes include B Cells, T Cells and NK (Natural Killer) cells. ALL is caused by Lymphoid Blasts, or Lymphoblasts, developing into immature Lymphocytes, and an abnormal amount of these immature Lymphocytes are produced. Lymphocytes are white blood cells and play a very important role in the immune system helping to fight off diseases. - -Acute Lymphoblastic Leukemia is most commonly found in children, and is the most common form of child cancer, with around 3000 cases a year in the US alone. Like Acute Myeloid Leukemia, although common, it is still quite rare. In both children and adults, early detection is critical. Treatment must start imassetstely due to the aggressiveness of the cancer. [More info](https://www.leukemiaresearchassociation.ai/leukemia). - -## ALL-IDB -You need to be granted access to use the Acute Lymphoblastic Leukemia Image Database for Image Processing dataset. You can find the application form and information about getting access to the dataset on [this page](https://homes.di.unimi.it/scotti/all/#download) as well as information on how to contribute back to the project [here](https://homes.di.unimi.it/scotti/all/results.php). If you are not able to obtain a copy of the dataset please feel free to try this tutorial on your own dataset, we would be very happy to find additional AML & ALL datasets. -### ALL_IDB1 -In this project, [ALL-IDB1](https://homes.di.unimi.it/scotti/all/#datasets) is used, one of the datsets from the Acute Lymphoblastic Leukemia Image Database for Image Processing dataset. We will use data augmentation to increase the amount of training and testing data we have. +[Acute lymphoblastic leukemia (ALL)](https://www.leukemiaairesearch.com/research/leukemia), also known as Acute Lymphocytic Leukemia, is a cancer that affects the lymphoid blood cell lineage. It is the most common leukemia in children, and it accounts for 10-20% of acute leukemias in adults. The prognosis for both adult and especially childhood ALL has improved substantially since the 1970s. The 5- year survival is approximately 95% in children. In adults, the 5-year survival varies between 25% and 75%, with more favorable results in younger than in older patients. -"The ALL_IDB1 version 1.0 can be used both for testing segmentation capability of algorithms, as well as the classification systems and image preprocessing methods. This dataset is composed of 108 images collected during September, 2005. It contains about 39000 blood elements, where the lymphocytes has been labeled by expert oncologists. The images are taken with different magnifications of the microscope ranging from 300 to 500." +For more information about Acute Lymphoblastic Leukemia please visit our [Leukemia Information Page](https://www.leukemiaairesearch.com/research/leukemia)   -# Acute Lymphoblastic Leukemia Tensorflow Classifier 2020 - -The Acute Lymphoblastic Leukemia Tensorflow Classifier 2020 network architecture is based on the proposed architecture in the [Acute Leukemia Classification Using Convolution Neural Network In Clinical Decision Support System](https://airccj.org/CSCP/vol7/csit77505.pdf "Acute Leukemia Classification Using Convolution Neural Network In Clinical Decision Support System") paper. The model was trained and tested on a selection of different CPUs/GPUs, with the **Intel® Core™ i7-7700HQ CPU @ 2.80GHz × 8** and **Windows 10** providing the most optimal results. - -| OS | Hardware | Training | Validation | Test | Accuracy | Recall | Precision | AUC/ROC | -| -------------------- | -------------------- | -------------------- | ----- | ---------- | ---------- | ---------- | ---------- | ---------- | -| Google Colab | Tesla K80 GPU | 1180 | 404 | 20 | 0.9727723 | 0.9727723 | 0.9727723 | 0.9948964 | -| Windows 10 | NVIDIA GeoForce GTX 1060 | 1180 | 404 | 20 | 0.97066015 | 0.97066015 | 0.97066015 | 0.9908836 | -| Ubuntu 18.04 | NVIDIA GTX 1050 Ti Ti/PCIe/SSE2 | 1180 | 404 | 20 | 0.97772276 | 0.97772276 | 0.97772276 | 0.9989155 | -| Ubuntu 18.04 | Intel® Core™ i7-7700HQ CPU @ 2.80GHz × 8 | 1180 | 404 | 20 | 0.9752475 | 0.9752475 | 0.9752475 | 0.991492 | -| Windows 10 | Intel® Core™ i7-7700HQ CPU @ 2.80GHz × 8 | 1180 | 404 | 20 | 0.9851485 | 0.9851485 | 0.9851485 | 0.9985846 | -| macOS Mojave 10.14.6 | Intel® Core™ i5 CPU @ 2.4 GHz | 1180 | 404 | 20 | 0.9589041 | 0.9589041 | 0.9589041 | 0.99483955 | +# ALL-IDB +You need to be granted access to use the Acute Lymphoblastic Leukemia Image Database for Image Processing dataset. You can find the application form and information about getting access to the dataset on [this page](https://homes.di.unimi.it/scotti/all/#download) as well as information on how to contribute back to the project [here](https://homes.di.unimi.it/scotti/all/results.php). If you are not able to obtain a copy of the dataset please feel free to try this tutorial on your own dataset, we would be very happy to find additional AML & ALL datasets. -Source: [Acute Lymphoblastic Leukemia Tensorflow Classifier 2020](https://github.com/AMLResearchProject/ALL-Tensorflow-2020#classifier) +## ALL_IDB1 +In this project, [ALL-IDB1](https://homes.di.unimi.it/scotti/all/#datasets) is used, one of the datsets from the Acute Lymphoblastic Leukemia Image Database for Image Processing dataset. We will use data augmentation to increase the amount of training and testing data we have.   -# Intel Technologies +# Intel® Technologies ## Intel® oneAPI Toolkits (Beta) [Intel® oneAPI Toolkits](https://software.intel.com/content/www/us/en/develop/tools/oneapi.html) are a collection of toolkits that provide the tools to optimize training and running inference on Artificial Intelligence models, maximizing the use of Intel architecture, including CPU, GPU, VPU and FPGA. @@ -107,9 +68,9 @@ The [Intel® Movidius™ Neural Compute Stick 2](https://software.intel.com/cont   -# Acute Lymphoblastic Leukemia oneAPI Classifier 2021 +# HIAS Acute Lymphoblastic Leukemia oneAPI Classifier -To create the newly improved **Acute Lymphoblastic Leukemia oneAPI Classifier 2021** we will take the following steps: +To create the newly improved **Acute Lymphoblastic Leukemia oneAPI Classifier** we will take the following steps: - Use the data augmentation techniques proposed in [the Leukemia Blood Cell Image Classification Using Convolutional Neural Network by T. T. P. Thanh, Caleb Vununu, Sukhrob Atoev, Suk-Hwan Lee, and Ki-Ryong Kwon](http://www.ijcte.org/vol10/1198-H0012.pdf). @@ -130,9 +91,9 @@ Ready to get started ? Head over to the [Getting Started guide](documentation/ge   # Contributing -The Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. +Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. -Please read the [CONTRIBUTING](CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page. +Please read the [CONTRIBUTING](CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find our code of conduct in the [Code of Conduct](CODE-OF-CONDUCT.md) document. ## Contributors - [Adam Milton-Barker](https://www.leukemiaairesearch.com/association/volunteers/adam-milton-barker "Adam Milton-Barker") - [Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain diff --git a/agent.py b/agent.py index cf956a9..da55077 100644 --- a/agent.py +++ b/agent.py @@ -28,76 +28,75 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-1 +- Adam Milton-Barker """ import sys -from abc import ABC, abstractmethod - from modules.AbstractAgent import AbstractAgent from modules.helpers import helpers from modules.server import server -from modules.model import model -#from modules.model_openvino import model_openvino class agent(AbstractAgent): - """ ALL oneAPI Classifier 2021 HIAS AI Agent - - Represents a HIAS AI Agent that processes data - using the ALL oneAPI Classifier 2021 model. - """ + """ HIAS ALL oneAPI Classifier - def train(self): - """ Creates & trains the model. """ + Represents a HIAS AI Agent that processes data + using the HIAS ALL oneAPI Classifier model. + """ - self.mqtt_start() + def train(self): + """ Creates & trains the model. """ - self.model.prepare_data() - self.model.prepare_network() - self.model.train() - self.model.evaluate() + self.mqtt_start() - def set_model(self, mtype): + self.model.prepare_data() + self.model.prepare_network() + self.model.train() + self.model.evaluate() - self.model_type = mtype - if self.model_type == "CNN": - self.model = model(self.helpers) - elif self.model_type == "IR": - self.model = model_openvino(self.helpers) + def set_model(self, model_type): - def load_model(self): - """ Loads the trained model """ + self.model_type = model_type + if self.model_type == "CNN": + from modules.model import model + self.model = model(self.helpers) + elif self.model_type == "IR": + from modules.model_openvino import model_openvino + self.model = model_openvino(self.helpers) - self.model.load() + def load_model(self): + """ Loads the trained model """ - def server(self): - """ Loads the API server """ + self.model.load() - self.mqtt_start() + def server(self): + """ Loads the API server """ - self.load_model() - self.server = server(self.helpers, self.model, self.model_type, self.mqtt) - self.server.start() + self.mqtt_start() + self.load_model() + self.server = server( + self.helpers, self.model, + self.model_type, self.mqtt) + self.server.start() - def inference(self): - """ Loads model and classifies test data locally """ + def inference(self): + """ Loads model and classifies test data locally """ - self.load_model() - self.model.test() + self.load_model() + self.model.test() - def inference_http(self): - """ Loads model and classifies test data via HTTP requests """ + def inference_http(self): + """ Loads model and classifies test data via HTTP requests """ - self.model.test_http() + self.model.test_http() - def signal_handler(self, signal, frame): - self.helpers.logger.info("Disconnecting") - self.mqtt.disconnect() - sys.exit(1) + def signal_handler(self, signal, frame): + self.helpers.logger.info("Disconnecting") + self.mqtt.disconnect() + sys.exit(1) agent = agent() @@ -105,43 +104,43 @@ def signal_handler(self, signal, frame): def main(): - if len(sys.argv) < 2: - print("You must provide an argument") - exit() - elif sys.argv[1] not in agent.helpers.confs["agent"]["params"]: - print("Mode not supported! server, train or inference") - exit() + if len(sys.argv) < 2: + print("You must provide an argument") + exit() + elif sys.argv[1] not in agent.helpers.confs["agent"]["params"]: + print("Mode not supported! server, train or inference") + exit() - mode = sys.argv[1] + mode = sys.argv[1] - if mode == "train": - agent.set_model("CNN") - agent.train() + if mode == "train": + agent.set_model("CNN") + agent.train() - elif mode == "classify": - agent.set_model("CNN") - agent.inference() + elif mode == "classify": + agent.set_model("CNN") + agent.inference() - elif mode == "server": - agent.set_model("CNN") - agent.server() + elif mode == "server": + agent.set_model("CNN") + agent.server() - elif mode == "classify_http": - agent.set_model("CNN") - agent.inference_http() + elif mode == "classify_http": + agent.set_model("CNN") + agent.inference_http() - elif mode == "classify_openvino": - agent.set_model("IR") - agent.inference() + elif mode == "classify_openvino": + agent.set_model("IR") + agent.inference() - elif mode == "server_openvino": - agent.set_model("IR") - agent.server() + elif mode == "server_openvino": + agent.set_model("IR") + agent.server() - elif mode == "classify_openvino_http": - agent.set_model("IR") - agent.inference_http() + elif mode == "classify_openvino_http": + agent.set_model("IR") + agent.inference_http() if __name__ == "__main__": - main() + main() diff --git a/assets/images/all-oneapi-classifier-2020.png b/assets/images/all-oneapi-classifier-2020.png deleted file mode 100644 index d4099e5..0000000 Binary files a/assets/images/all-oneapi-classifier-2020.png and /dev/null differ diff --git a/assets/images/bug-report.png b/assets/images/bug-report.jpg similarity index 100% rename from assets/images/bug-report.png rename to assets/images/bug-report.jpg diff --git a/assets/images/feature-proposals.png b/assets/images/feature-proposals.jpg similarity index 100% rename from assets/images/feature-proposals.png rename to assets/images/feature-proposals.jpg diff --git a/assets/images/feature-request.png b/assets/images/feature-request.jpg similarity index 100% rename from assets/images/feature-request.png rename to assets/images/feature-request.jpg diff --git a/assets/images/fork.png b/assets/images/fork.jpg similarity index 100% rename from assets/images/fork.png rename to assets/images/fork.jpg diff --git a/assets/images/hias-ai-agent-edit.jpg b/assets/images/hias-ai-agent-edit.jpg deleted file mode 100644 index 5ea8cde..0000000 Binary files a/assets/images/hias-ai-agent-edit.jpg and /dev/null differ diff --git a/assets/images/hias-ai-agent.jpg b/assets/images/hias-ai-agent.jpg deleted file mode 100644 index 17ce792..0000000 Binary files a/assets/images/hias-ai-agent.jpg and /dev/null differ diff --git a/assets/images/hias-ai-inference-endpoint.jpg b/assets/images/hias-ai-inference-endpoint.jpg deleted file mode 100644 index 7437d6c..0000000 Binary files a/assets/images/hias-ai-inference-endpoint.jpg and /dev/null differ diff --git a/assets/images/hias-ai-inference.jpg b/assets/images/hias-ai-inference.jpg deleted file mode 100644 index 1fb0ca7..0000000 Binary files a/assets/images/hias-ai-inference.jpg and /dev/null differ diff --git a/assets/images/hias-ai-model-edit.jpg b/assets/images/hias-ai-model-edit.jpg deleted file mode 100644 index 3049155..0000000 Binary files a/assets/images/hias-ai-model-edit.jpg and /dev/null differ diff --git a/assets/images/hias-ai-model.jpg b/assets/images/hias-ai-model.jpg deleted file mode 100644 index 6885fde..0000000 Binary files a/assets/images/hias-ai-model.jpg and /dev/null differ diff --git a/assets/images/project-banner.jpg b/assets/images/project-banner.jpg new file mode 100644 index 0000000..e8336ea Binary files /dev/null and b/assets/images/project-banner.jpg differ diff --git a/assets/images/repo-issues.png b/assets/images/repo-issues.jpg similarity index 100% rename from assets/images/repo-issues.png rename to assets/images/repo-issues.jpg diff --git a/configuration/config.json b/configuration/config.json index ae479b3..ddf1c68 100644 --- a/configuration/config.json +++ b/configuration/config.json @@ -1,8 +1,6 @@ { "agent": { "cores": 8, - "server": "", - "port": 1234, "params": [ "train", "classify", diff --git a/docs/img/hias-ai-agent-edit.jpg b/docs/img/hias-ai-agent-edit.jpg new file mode 100644 index 0000000..a1db15a Binary files /dev/null and b/docs/img/hias-ai-agent-edit.jpg differ diff --git a/docs/img/hias-ai-agent.jpg b/docs/img/hias-ai-agent.jpg new file mode 100644 index 0000000..1e1950b Binary files /dev/null and b/docs/img/hias-ai-agent.jpg differ diff --git a/docs/img/hias-ai-inference-classification.jpg b/docs/img/hias-ai-inference-classification.jpg new file mode 100644 index 0000000..d169469 Binary files /dev/null and b/docs/img/hias-ai-inference-classification.jpg differ diff --git a/docs/img/hias-ai-inference-endpoint.jpg b/docs/img/hias-ai-inference-endpoint.jpg new file mode 100644 index 0000000..269de55 Binary files /dev/null and b/docs/img/hias-ai-inference-endpoint.jpg differ diff --git a/docs/img/hias-ai-inference.jpg b/docs/img/hias-ai-inference.jpg new file mode 100644 index 0000000..bc2646a Binary files /dev/null and b/docs/img/hias-ai-inference.jpg differ diff --git a/docs/img/hias-ai-model-edit.jpg b/docs/img/hias-ai-model-edit.jpg new file mode 100644 index 0000000..b0f1430 Binary files /dev/null and b/docs/img/hias-ai-model-edit.jpg differ diff --git a/docs/img/hias-ai-model.jpg b/docs/img/hias-ai-model.jpg new file mode 100644 index 0000000..768dc05 Binary files /dev/null and b/docs/img/hias-ai-model.jpg differ diff --git a/docs/img/plots/accuracy.png b/docs/img/plots/accuracy.png new file mode 100644 index 0000000..9ed0a5e Binary files /dev/null and b/docs/img/plots/accuracy.png differ diff --git a/docs/img/plots/auc.png b/docs/img/plots/auc.png new file mode 100644 index 0000000..3f1b824 Binary files /dev/null and b/docs/img/plots/auc.png differ diff --git a/docs/img/plots/confusion-matrix.png b/docs/img/plots/confusion-matrix.png new file mode 100644 index 0000000..031c66a Binary files /dev/null and b/docs/img/plots/confusion-matrix.png differ diff --git a/docs/img/plots/loss.png b/docs/img/plots/loss.png new file mode 100644 index 0000000..bb6a304 Binary files /dev/null and b/docs/img/plots/loss.png differ diff --git a/docs/img/plots/precision.png b/docs/img/plots/precision.png new file mode 100644 index 0000000..832ed33 Binary files /dev/null and b/docs/img/plots/precision.png differ diff --git a/docs/img/plots/recall.png b/docs/img/plots/recall.png new file mode 100644 index 0000000..371738e Binary files /dev/null and b/docs/img/plots/recall.png differ diff --git a/docs/img/project-banner.jpg b/docs/img/project-banner.jpg new file mode 100644 index 0000000..e8336ea Binary files /dev/null and b/docs/img/project-banner.jpg differ diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..0ee3d16 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,66 @@ +# Documentation + +![Acute Lymphoblastic Leukemia oneAPI Classifier](img/project-banner.jpg) + +# Welcome + +Welcome to the [HIAS Acute Lymphoblastic Leukemia oneAPI Classifier](https://github.com/aiial/hias-all-oneapi-classifier) official documentation. + +  + +# Introduction + +The **Acute Lymphoblastic Leukemia (ALL) oneAPI Classifier** is an open-source classifier programmed using the [Intel® Distribution for Python*](https://software.intel.com/content/www/us/en/develop/tools/distribution-for-python.html) and trained using [Intel® Optimization for TensorFlow*](https://software.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation-guide.html). The model is deployed on a Raspberry 4 using [Intel® Distribution of OpenVINO™ Toolkit](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html) and inference is carried out using the [Intel® Neural Compute Stick 2 (Intel® NCS2)](https://software.intel.com/content/www/us/en/develop/hardware/neural-compute-stick.html). + +  + +# DISCLAIMER + +_This project should be used for research purposes only. The purpose of the project is to show the potential of Artificial Intelligence for medical support systems such as diagnostic systems._ + +_Although the model is accurate and shows good results both on paper and in real world testing, it is trained on a small amount of data and needs to be trained on larger datasets to really evaluate it's accuracy._ + +_Developers that have contributed to this repository have experience in using Artificial Intelligence for detecting certain types of cancer. They are not doctors, medical or cancer experts._ + +  + +# Installation + +Use the following installation guides to set up your project.: + +- [Ubuntu Installation Guide](installation/ubuntu.md) +- [Raspberry Pi 4 Installation Guide](installation/raspberry-pi.md) + +  + +# Usage + +Use the following usage guides to to train your classifier and use your classifier on Arduino: + +- [Ubuntu Usage Guide](usage/ubuntu.md) +- [Raspberry Pi 4 Usage Guide](usage/raspberry-pi.md) + +  + +# Contributing +Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss encourages and welcomes code contributions, bug fixes and enhancements from the Github community. + +Please read the [CONTRIBUTING](https://github.com/aiial/hias-all-oneapi-classifier/blob/main/CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find our code of conduct in the [Code of Conduct](https://github.com/aiial/hias-all-oneapi-classifier/blob/main/CODE-OF-CONDUCT.md) document. + +## Contributors +- [Adam Milton-Barker](https://www.leukemiaairesearch.com/association/volunteers/adam-milton-barker "Adam Milton-Barker") - [Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain + +  + +# Versioning +We use SemVer for versioning. + +  + +# License +This project is licensed under the **MIT License** - see the [LICENSE](https://github.com/aiial/hias-all-oneapi-classifier/blob/main/LICENSE "LICENSE") file for details. + +  + +# Bugs/Issues +We use the [repo issues](https://github.com/aiial/hias-all-oneapi-classifier/blob/main/issues "repo issues") to track bugs and general requests related to using this project. See [CONTRIBUTING](https://github.com/aiial/hias-all-oneapi-classifier/blob/main/CONTRIBUTING.md "CONTRIBUTING") for more info on how to submit bugs, feature requests and proposals. \ No newline at end of file diff --git a/docs/installation/raspberry-pi.md b/docs/installation/raspberry-pi.md new file mode 100644 index 0000000..95a5d54 --- /dev/null +++ b/docs/installation/raspberry-pi.md @@ -0,0 +1,92 @@ +# Installation (Raspberry Pi) + +![Acute Lymphoblastic Leukemia oneAPI Classifier](../img/project-banner.jpg) + +# Introduction +This guide will guide you through the installation process for the **HIAS Acute Lymphoblastic Leukemia oneAPI Classifier** on your Ubuntu development machine. + +  + +# Raspberry Pi OS +For this Project, the operating system choice is [Raspberry Pi OS](https://www.raspberrypi.org/downloads/raspberry-pi-os/ "Raspberry Pi OS"). + +  + +# Intel® Distribution of OpenVINO™ Toolkit +To install Intel® Distribution of OpenVINO™ Toolkit for Raspberry Pi, navigate to the home directory on your Raspberry Pi and use the following commands: + +``` + wget https://download.01.org/opencv/2021/openvinotoolkit/2021.2/l_openvino_toolkit_runtime_raspbian_p_2021.2.185.tgz +``` +``` + sudo mkdir -p /opt/intel/openvino + sudo tar -xf l_openvino_toolkit_runtime_raspbian_p_2021.2.185.tgz --strip 1 -C /opt/intel/openvino +``` +``` + sudo apt install cmake + source /opt/intel/openvino/bin/setupvars.sh + echo "source /opt/intel/openvino/bin/setupvars.sh" >> ~/.bashrc +``` + +  + +# Intel® Movidius™ Neural Compute Stick 2 +Now we will set up ready for Neural Compute Stick 2. +``` + sudo usermod -a -G users "$(whoami)" +``` +Now close your existing terminal and open a new open. Once in your new terminal use the following commands: +``` + sh /opt/intel/openvino/install_dependencies/install_NCS_udev_rules.sh +``` + +  + +# Transfer Files + +Next you need to transfer the project folder to your Raspberry Pi, make sure that you have all of the files from the model directory. + +  + +# Software Install + +All other requirements are included in **scripts/install-rpi4.sh**. You can run this file on machine by navigating to the project root in terminal and using the commands below: + +``` + sh scripts/install-rpi4.sh +``` + +  + +# Continue +Now you can continue with the Acute Lymphoblastic Leukemia oneAPI Classifier [Raspberry Pi 4 usage guide](../usage/raspberry-pi.md). + +  + +# Contributing + +The Peter Moss Acute Myeloid & Lymphoblastic Leukemia AI Research project encourages and youlcomes code contributions, bug fixes and enhancements from the Github. + +Please read the [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page. + +## Contributors + +- [Adam Milton-Barker](https://www.leukemiaresearchassociation.ai/team/adam-milton-barker "Adam Milton-Barker") - [Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain + +  + +# Versioning + +You use SemVer for versioning. For the versions available, see [Releases](../../releases "Releases"). + +  + +# License + +This project is licensed under the **MIT License** - see the [LICENSE](../../LICENSE "LICENSE") file for details. + +  + +# Bugs/Issues + +You use the [repo issues](../../issues "repo issues") to track bugs and general requests related to using this project. See [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") for more info on how to submit bugs, feature requests and proposals. \ No newline at end of file diff --git a/documentation/installation/ubuntu.md b/docs/installation/ubuntu.md similarity index 50% rename from documentation/installation/ubuntu.md rename to docs/installation/ubuntu.md index ebebd5d..5ef3270 100644 --- a/documentation/installation/ubuntu.md +++ b/docs/installation/ubuntu.md @@ -1,34 +1,11 @@ -# Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss -## Acute Lymphoblastic Leukemia oneAPI Classifier -### Ubuntu Installation +# Installation (Ubuntu) -![Acute Lymphoblastic Leukemia oneAPI Classifier](../../assets/images/all-oneapi-classifier-2020.png) - -  - -# Table Of Contents - -- [Introduction](#introduction) -- [Prerequisites](#prerequisites) - - [Anaconda](#anaconda) -- [Installation](#installation) - - [Intel® Optimization for TensorFlow](#intel-optimization-for-tensorflow) - - [Intel® Distribution of OpenVINO™ Toolkit](#intel-distribution-of-openvino-toolkit) - - [Clone The Repository](#clone-the-repository) - - [Developer Forks](#developer-forks) - - [Setup File](#setup-file) -- [HIAS](#hias) - - [AI Model](#ai-model) - - [AI Agent](#ai-agent) -- [Continue](#continue) -- [Contributing](#contributing) - - [Contributors](#contributors) -- [Versioning](#versioning) -- [License](#license) -- [Bugs/Issues](#bugs-issues) +![Acute Lymphoblastic Leukemia oneAPI Classifier](../img/project-banner.jpg) # Introduction -This guide will guide you through the installation process for the Acute Lymphoblastic Leukemia oneAPI Classifier. +This guide will take you through the installation process for the **HIAS Acute Lymphoblastic Leukemia oneAPI Classifier** on your Ubuntu development machine. + +  # Prerequisites You will need to ensure you have the following prerequisites installed and setup. @@ -36,72 +13,82 @@ You will need to ensure you have the following prerequisites installed and setup ## Anaconda If you haven't already installed Anaconda you will need to install it now. Follow the [Anaconda installation guide](https://docs.anaconda.com/anaconda/install/ "Anaconda installation guide") to do so. -## HIAS Server -For this project you will need a functioning [HIAS Server](https://github.com/AIIAL/HIAS). To install the HIAS Server, follow the [HIAS Server Installation Guide](https://github.com/AIIAL/HIAS/blob/master/Documentation/Installation/Installation.md) +## HIAS Core +For this project you will need a functioning [HIAS Core](https://github.com/aiial/hias-core). To install HIAS Core follow the [HIAS Core Installation Guide](https://hias-core.readthedocs.io/en/latest/installation/ubuntu/) -# Installation -You are now ready to install the Acute Lymphoblastic Leukemia oneAPI Classifier software. +  -## Intel® Optimization for TensorFlow -Now you will install the Intel® Optimization for TensorFlow using Anaconda. +# Intel® oneAPI Basekit +First you will install Intel® OneAPI basekit: -``` -conda create -n all-tfmkl python=3 -conda activate all-tfmkl +``` bash +conda create -n hias-all-oneapi-classifier python=3 +conda activate hias-all-oneapi-classifier conda install tensorflow -c anaconda -conda deactivate ``` -## Intel® Distribution of OpenVINO™ Toolkit +  + +# Intel® Distribution of OpenVINO™ Toolkit Now you will install Intel® Distribution of OpenVINO™ Toolkit which will be used to convert your frozen model into an Intermediate Representation. -``` -conda activate all-tfmkl +## Ubuntu 18.04 + +``` bash conda install openvino-ie4py-ubuntu18 -c intel -pip install test-generator==0.1.1 +conda install -c conda-forge defusedxml +pip3 install test-generator==0.1.1 ``` -## Clone the repository - -Clone the [Acute Lymphoblastic Leukemia oneAPI Classifier](https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier " Acute Lymphoblastic Leukemia oneAPI Classifier") repository from the [Peter Moss Acute Myeloid & Lymphoblastic Leukemia AI Research Project](https://github.com/AIIAL "Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss") Github Organization. -To clone the repository and install the OneAPI Acute Lymphoblastic Leukemia Classifier Classifier, make sure you have Git installed. Now navigate to the a directory on your device using commandline, and then use the following command. +## Ubuntu 20.04 +``` bash +conda install openvino-ie4py-ubuntu20 -c intel +conda install -c conda-forge defusedxml +pip3 install test-generator==0.1.1 ``` - git clone https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier.git -``` -Once you have used the command above you will see a directory called **Acute-Lymphoblastic-Leukemia-oneAPI-Classifier** in your home directory. +  + +# Clone the repository +Clone the [HIAS Acute Lymphoblastic Leukemia oneAPI Classifier](https://github.com/aiial/hias-all-oneapi-classifier " HIAS Acute Lymphoblastic Leukemia oneAPI Classifier") repository from the [Peter Moss MedTech Research Project](https://github.com/aiial "Peter Moss MedTech Research Project") Github Organization. + +To clone the repository make sure you have Git installed. Now navigate to the a directory on your device using commandline, and then use the following command. + +``` bash + git clone https://github.com/aiial/hias-all-oneapi-classifier.git ``` + +Once you have used the command above you will see a directory called **hias-all-oneapi-classifier** in your home directory. + +``` bash ls ``` Using the ls command in your home directory should show you the following. +``` bash + hias-all-oneapi-classifier ``` - Acute-Lymphoblastic-Leukemia-oneAPI-Classifier -``` - -Navigate to the **Acute-Lymphoblastic-Leukemia-oneAPI-Classifier** directory, this is your project root directory for this tutorial. -### Developer Forks +Navigate to the **hias-all-oneapi-classifier** directory, this is your project root directory for this tutorial. -Developers from the Github community that would like to contribute to the development of this project should first create a fork, and clone that repository. For detailed information please view the [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") guide. You should pull the latest code from the development branch. +  -``` - git clone -b "1.1.0" https://github.com/AIIAL/Acute-Lymphoblastic-Leukemia-oneAPI-Classifier.git -``` +# Installation File -The **-b "1.1.0"** parameter ensures you get the code from the latest master branch. Before using the below command please check our latest master branch in the button at the top of the project README. +All other software requirements are included in **scripts/install.sh**. You can run this file on your machine from the project root in terminal. Ensuring your `hias-all-oneapi-classifier` conda environment is active, use the commands that follow: -## Setup File +``` bash + sh scripts/install.sh +``` -All other software requirements are included in **scripts/install.sh**. You can run this file on your machine from the project root in terminal. Use the commands that follow: +If you receive any errors after running the above command, stop the installation and run the following commands: -``` - conda activate all-tfmkl +``` bash + sed -i 's/\r//' scripts/install.sh sh scripts/install.sh - conda deactivate ```   @@ -112,7 +99,7 @@ This device is a HIAS AI Agent and uses the HIAS MQTT Broker to communicate with The HIAS network is powered by a context broker that stores contextual data and exposes the data securely to authenticated HIAS applications and devices. -Each HIAS AI Agent & AI Model has a JSON representation stored in the HIAS Context Broker that holds their contextual information. +Each HIAS AI Agent & AI Model has a JSON representation stored in the HIASCDI Context Broker that holds their contextual information. ## AI Model @@ -120,23 +107,23 @@ A HIAS AI Model is a JSON representation of an Artificial Intelligence model use First you need to set a HIAS AI Model up in the HIAS UI. Navigate to **AI->Models->Create** to create a HIAS AI Model. A future release of HIAS will provide the functionality to import the HIAS JSON representation of the AI Model, but for now you have to manually create the AI Model in the UI. -![HIAS AI Model](../../assets/images/hias-ai-model.jpg) +![HIAS AI Model](../img/hias-ai-model.jpg) Once you have completed the form and submitted it, you can find the newly created AI Model by navigating to **AI->Models->List** and clicking on the relevant Model. On the HIAS AI Model page you will be able to update the contextual data for the model, and also find the JSON representation. -![HIAS AI Model](../../assets/images/hias-ai-model-edit.jpg) +![HIAS AI Model](../img/hias-ai-model-edit.jpg) ## AI Agent A HIAS AI Agent is a bridge between HIAS devices and applications, and HIAS IoT Agents. The AI Agents process incoming data by passing it through HIAS AI Models and returning the response back to the requesting device/application. -As with AI Models, AI Agents have an entry in the HIAS Context Broker and a JSON representation stored on the network. +As with AI Models, AI Agents have an entry in the HIASCDI Context Broker and a JSON representation stored on the network. You will now need to create your HIAS AI Agent and retrieve the credentials required by your Acute Lymphoblastic Leukemia oneAPI Classifier. Navigate to **AI->Agents->Create** to create a HIAS AI Model. -![HIAS AI Agent](../../assets/images/hias-ai-agent.jpg) +![HIAS AI Agent](../img/hias-ai-agent.jpg) **MAKE SURE YOU SELECT THE PREVIOUSLY CREATED HIAS AI MODEL** @@ -144,20 +131,16 @@ Once you have completed the form and submitted it, you can find the newly create On the HIAS AI Agent page you will be able to update the contextual data for the agent, and also find the JSON representation. -![HIAS AI Agent](../../assets/images/hias-ai-agent-edit.jpg) +![HIAS AI Agent](../img/hias-ai-agent-edit.jpg) You now need to download the credentials required to connect the Acute Lymphoblastic Leukemia oneAPI Classifier to the HIAS network. -Click on the **Agent Credentials** section to download the credentials file. This should open your file browser, navigate to the **Acute-Lymphoblastic-Leukemia-oneAPI-Classifier/configuration/** directory and save the file as **credentials.json**. +Click on the **Agent Credentials** button to download the credentials file. This should open your file browser, navigate to the **hias-all-oneapi-classifier/configuration/** directory and save the file as **credentials.json**.   # Continue -When you are ready to continue, activate your Anaconda environment: -``` - conda activate all-tfmkl -``` -Now you can continue with the Acute Lymphoblastic Leukemia oneAPI Classifier [getting started guide](../getting-started.md) +Now you can continue with the Acute Lymphoblastic Leukemia oneAPI Classifier [usage guide](../usage/ubuntu.md) to train your model.   diff --git a/docs/usage/raspberry-pi.md b/docs/usage/raspberry-pi.md new file mode 100644 index 0000000..44e742f --- /dev/null +++ b/docs/usage/raspberry-pi.md @@ -0,0 +1,222 @@ +# Usage (Raspberry Pi) + +![Acute Lymphoblastic Leukemia oneAPI Classifier](../img/project-banner.jpg) + +# Introduction +This guide will take you through using the **HIAS Acute Lymphoblastic Leukemia oneAPI Classifier** on your Raspberry Pi 4. + +  + +# Local Testing + +Now you will test the classifier on your Raspberry Pi 4 using OpenVINO & Neural Compute Stick 2. You will use the 20 images that were removed from the training data during installation. + +To run the AI Agent in test mode use the following command: + +``` bash +python3 agent.py classify_openvino +``` + +You should see the application will start processing the test images and the results will be displayed in the console. + +``` bash +2021-09-19 00:31:56,143 - Agent - INFO - Loaded test image model/data/test/Im099_0.jpg +2021-09-19 00:31:57,910 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 1.7643792629241943 seconds. +2021-09-19 00:31:58,233 - Agent - INFO - Loaded test image model/data/test/Im047_0.jpg +2021-09-19 00:31:58,248 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.013955831527709961 seconds. +2021-09-19 00:31:58,570 - Agent - INFO - Loaded test image model/data/test/Im106_0.jpg +2021-09-19 00:31:58,586 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.013903617858886719 seconds. +2021-09-19 00:31:58,725 - Agent - INFO - Loaded test image model/data/test/Im020_1.jpg +2021-09-19 00:31:58,741 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.013935089111328125 seconds. +2021-09-19 00:31:59,036 - Agent - INFO - Loaded test image model/data/test/Im095_0.jpg +2021-09-19 00:31:59,051 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 0.01394963264465332 seconds. +2021-09-19 00:31:59,375 - Agent - INFO - Loaded test image model/data/test/Im053_1.jpg +2021-09-19 00:31:59,391 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in 0.013952493667602539 seconds. +2021-09-19 00:31:59,530 - Agent - INFO - Loaded test image model/data/test/Im024_1.jpg +2021-09-19 00:31:59,546 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.013888835906982422 seconds. +2021-09-19 00:31:59,866 - Agent - INFO - Loaded test image model/data/test/Im069_0.jpg +2021-09-19 00:31:59,882 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.013928413391113281 seconds. +2021-09-19 00:32:00,201 - Agent - INFO - Loaded test image model/data/test/Im074_0.jpg +2021-09-19 00:32:00,217 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.013946533203125 seconds. +2021-09-19 00:32:00,356 - Agent - INFO - Loaded test image model/data/test/Im006_1.jpg +2021-09-19 00:32:00,371 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.013928890228271484 seconds. +2021-09-19 00:32:00,694 - Agent - INFO - Loaded test image model/data/test/Im088_0.jpg +2021-09-19 00:32:00,710 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.014078855514526367 seconds. +2021-09-19 00:32:01,032 - Agent - INFO - Loaded test image model/data/test/Im060_1.jpg +2021-09-19 00:32:01,061 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.013947486877441406 seconds. +2021-09-19 00:32:01,383 - Agent - INFO - Loaded test image model/data/test/Im101_0.jpg +2021-09-19 00:32:01,399 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.01399540901184082 seconds. +2021-09-19 00:32:01,537 - Agent - INFO - Loaded test image model/data/test/Im031_1.jpg +2021-09-19 00:32:01,553 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.014025211334228516 seconds. +2021-09-19 00:32:01,874 - Agent - INFO - Loaded test image model/data/test/Im063_1.jpg +2021-09-19 00:32:01,889 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.014019250869750977 seconds. +2021-09-19 00:32:02,211 - Agent - INFO - Loaded test image model/data/test/Im035_0.jpg +2021-09-19 00:32:02,227 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.014011383056640625 seconds. +2021-09-19 00:32:02,549 - Agent - INFO - Loaded test image model/data/test/Im057_1.jpg +2021-09-19 00:32:02,564 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.01399540901184082 seconds. +2021-09-19 00:32:02,886 - Agent - INFO - Loaded test image model/data/test/Im041_0.jpg +2021-09-19 00:32:02,902 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.014095306396484375 seconds. +2021-09-19 00:32:03,041 - Agent - INFO - Loaded test image model/data/test/Im028_1.jpg +2021-09-19 00:32:03,057 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.013976573944091797 seconds. +2021-09-19 00:32:03,194 - Agent - INFO - Loaded test image model/data/test/Im026_1.jpg +2021-09-19 00:32:03,210 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.014018535614013672 seconds. +2021-09-19 00:32:03,210 - Agent - INFO - Images Classifier: 20 +2021-09-19 00:32:03,210 - Agent - INFO - True Positives: 9 +2021-09-19 00:32:03,211 - Agent - INFO - False Positives: 1 +2021-09-19 00:32:03,211 - Agent - INFO - True Negatives: 9 +2021-09-19 00:32:03,211 - Agent - INFO - False Negatives: 1 +2021-09-19 00:32:03,211 - Agent - INFO - Total Time Taken: 2.0299320220947266 +``` + +  + +# Server Testing + +In the current terminal, now use the following command: + +``` bash +python3 agent.py server_openvino +``` + +This will start the server on your Raspberry Pi that exposes the model via a REST API. Now open a new terminal, navigate to the project root and use the following command: + +``` bash +python3 agent.py classify_openvino_http +``` + +This will start agent in HTTP Inference mode. The agent will loop through the testing data and send each image to the server for classification, the results are then displayed in the console. + +``` bash +2021-09-19 00:42:25,855 - Agent - INFO - Sending request for: model/data/test/Im099_0.jpg +2021-09-19 00:42:28,952 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 3.096384048461914 seconds. +2021-09-19 00:42:28,959 - Agent - INFO - Sending request for: model/data/test/Im047_0.jpg +2021-09-19 00:42:30,313 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3539931774139404 seconds. +2021-09-19 00:42:30,314 - Agent - INFO - Sending request for: model/data/test/Im106_0.jpg +2021-09-19 00:42:31,663 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3494203090667725 seconds. +2021-09-19 00:42:31,663 - Agent - INFO - Sending request for: model/data/test/Im020_1.jpg +2021-09-19 00:42:32,334 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.671001672744751 seconds. +2021-09-19 00:42:32,335 - Agent - INFO - Sending request for: model/data/test/Im095_0.jpg +2021-09-19 00:42:33,590 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 1.255136251449585 seconds. +2021-09-19 00:42:33,590 - Agent - INFO - Sending request for: model/data/test/Im053_1.jpg +2021-09-19 00:42:34,948 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Negative) in 1.357445478439331 seconds. +2021-09-19 00:42:34,948 - Agent - INFO - Sending request for: model/data/test/Im024_1.jpg +2021-09-19 00:42:35,589 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6405436992645264 seconds. +2021-09-19 00:42:35,589 - Agent - INFO - Sending request for: model/data/test/Im069_0.jpg +2021-09-19 00:42:36,936 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3462553024291992 seconds. +2021-09-19 00:42:36,936 - Agent - INFO - Sending request for: model/data/test/Im074_0.jpg +2021-09-19 00:42:38,276 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3395044803619385 seconds. +2021-09-19 00:42:38,276 - Agent - INFO - Sending request for: model/data/test/Im006_1.jpg +2021-09-19 00:42:38,924 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6482028961181641 seconds. +2021-09-19 00:42:38,924 - Agent - INFO - Sending request for: model/data/test/Im088_0.jpg +2021-09-19 00:42:40,274 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3498878479003906 seconds. +2021-09-19 00:42:40,275 - Agent - INFO - Sending request for: model/data/test/Im060_1.jpg +2021-09-19 00:42:41,656 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 1.3813292980194092 seconds. +2021-09-19 00:42:41,657 - Agent - INFO - Sending request for: model/data/test/Im101_0.jpg +2021-09-19 00:42:43,043 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3861136436462402 seconds. +2021-09-19 00:42:43,043 - Agent - INFO - Sending request for: model/data/test/Im031_1.jpg +2021-09-19 00:42:43,717 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6737401485443115 seconds. +2021-09-19 00:42:43,717 - Agent - INFO - Sending request for: model/data/test/Im063_1.jpg +2021-09-19 00:42:45,064 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 1.347097396850586 seconds. +2021-09-19 00:42:45,072 - Agent - INFO - Sending request for: model/data/test/Im035_0.jpg +2021-09-19 00:42:46,423 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.35062575340271 seconds. +2021-09-19 00:42:46,423 - Agent - INFO - Sending request for: model/data/test/Im057_1.jpg +2021-09-19 00:42:47,769 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 1.3457365036010742 seconds. +2021-09-19 00:42:47,770 - Agent - INFO - Sending request for: model/data/test/Im041_0.jpg +2021-09-19 00:42:49,115 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3452880382537842 seconds. +2021-09-19 00:42:49,115 - Agent - INFO - Sending request for: model/data/test/Im028_1.jpg +2021-09-19 00:42:49,787 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6719515323638916 seconds. +2021-09-19 00:42:49,788 - Agent - INFO - Sending request for: model/data/test/Im026_1.jpg +2021-09-19 00:42:50,433 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.644808292388916 seconds. +2021-09-19 00:42:50,433 - Agent - INFO - Images Classified: 20 +2021-09-19 00:42:50,433 - Agent - INFO - True Positives: 9 +2021-09-19 00:42:50,433 - Agent - INFO - False Positives: 1 +2021-09-19 00:42:50,434 - Agent - INFO - True Negatives: 9 +2021-09-19 00:42:50,434 - Agent - INFO - False Negatives: 1 +2021-09-19 00:42:50,434 - Agent - INFO - Total Time Taken: 24.554465770721436 +``` + +  + +# HIAS UI + +Now that your classifier is set up and running, you can interact with it via the HIAS UI, and from other HIAS integrated applications. Before you can do so there is a final step to take on your server. + +![HIAS AI Inference](../img/hias-ai-inference-endpoint.jpg) + +Head to the AI Agent page for your classifier on HIAS **(AI->Agents->List->Your Agent)**. On the edit page you will see the **Inference Endpoint**, you need to copy that value. + +## NGINX + +Now in console open the NGINX config file: + +``` bash +sudo nano /etc/nginx/sites-available/default +``` + +Find **ADD NEW ENDPOINTS AFTER THIS NOTICE**, and add the following, replacing **YourEndpoint** with your inference endpoint value, and **YourIp/YourPort** with the IP/port of your Raspberry Pi. + +``` bash +location ~ ^/AI/YourEndpoint/(.*)$ { + auth_basic "Restricted"; + auth_basic_user_file /etc/nginx/security/htpasswd; + proxy_pass http://YourIp:YourPort/$1; +} +``` + +Save the file and exit, then run the following command: + +``` bash +sudo systemctl reload nginx +``` + +## Inference + +Now you are set up to communicate with the HIASsAcute Lymphoblastic Leukemia oneAPI Classifier from the HIAS UI. Head to **(AI->Agents->List)** and then click on the **Diagnose** link on the relevant AI Agent. + +![HIAS AI Inference](../img/hias-ai-inference.jpg) + +Once on the inference UI upload the twenty test images. Now make sure the server is running on the RPI and click the data to send it to the Acute Lymphoblastic Leukemia oneAPI Classifier for classification. + +![HIAS AI Inference](../img/hias-ai-inference-classification.jpg) + +### Verification + +As we know from the filenames in advance whether an image is negative or positive, we can compare the classification with the file name to check if a classification is a true/false positive, or a true/false negative. In the Diagnosis Results area Diagnosis represents the classification provided by the Acute Lymphoblastic Leukemia oneAPI Classifier, and Result provides the verification result. You should get the same results as when testing earlier back in the tutorial. The UI should have one false negative and one false positive. + +  + +# Compatible Projects +This classifier can power a number of HIAS compatible projects. Below are the direct links to the projects that are compatible with this classifier. + +- [ALL Detection System For Oculus Rift](https://github.com/aiial/all-detection-system-for-oculus-rift) +- [ALL Detection System For Magic Leap 1](https://github.com/aiial/all-detection-system-for-magic-leap-1) + +  + +# Contributing + +The Peter Moss Acute Myeloid & Lymphoblastic Leukemia AI Research project encourages and youlcomes code contributions, bug fixes and enhancements from the Github. + +Please read the [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page. + +## Contributors + +- [Adam Milton-Barker](https://www.leukemiaresearchassociation.ai/team/adam-milton-barker "Adam Milton-Barker") - [Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain + +  + +# Versioning + +You use SemVer for versioning. For the versions available, see [Releases](../../releases "Releases"). + +  + +# License + +This project is licensed under the **MIT License** - see the [LICENSE](../../LICENSE "LICENSE") file for details. + +  + +# Bugs/Issues + +You use the [repo issues](../../issues "repo issues") to track bugs and general requests related to using this project. See [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") for more info on how to submit bugs, feature requests and proposals. \ No newline at end of file diff --git a/docs/usage/ubuntu.md b/docs/usage/ubuntu.md new file mode 100644 index 0000000..fbc8ff3 --- /dev/null +++ b/docs/usage/ubuntu.md @@ -0,0 +1,380 @@ +# Usage (Ubuntu) + +![Acute Lymphoblastic Leukemia oneAPI Classifier](../img/project-banner.jpg) + +# Introduction +This guide will take you through training the **HIAS Acute Lymphoblastic Leukemia oneAPI Classifier** on your Ubuntu development machine. + +  + +# Installation +Make sure you have completed the [Ubuntu installation guide](../installation/ubuntu.md) before you continue with this tutorial. + +  + +# Data +You need to be granted access to use the Acute Lymphoblastic Leukemia Image Database for Image Processing dataset. You can find the application form and information about getting access to the dataset on [this page](https://homes.di.unimi.it/scotti/all/#download) as well as information on how to contribute back to the project [here](https://homes.di.unimi.it/scotti/all/results.php). + +Once you have your data you need to add it to the project filesystem. You will notice the data folder in the `model` directory, **model/data**, inside you have **train** & **test**. Add all of the images from the ALL_IDB1 dataset to the **model/data/train** folder. + +## Application testing data + +In the data processing stage, ten negative images and ten positive images are removed from the dataset and moved to the **model/data/test/** directory. This data is not seen by the network during the training process, and is used for testing and by related applications. + +To ensure your model gets the same results, please use the same test images. By default HIAS compatible projects will be expecting the same test images. You can also try with your own image selection, however results may vary and you will need to make additional changes to our HIAS compatible projects. + +To specify which test images to use modify the [configuration/config.json](../../configuration/config.json) file as shown below: + +``` +"test_data": [ + "im006_1.jpg", + "im020_1.jpg", + "im024_1.jpg", + "im026_1.jpg", + "im028_1.jpg", + "im031_1.jpg", + "im035_0.jpg", + "im041_0.jpg", + "im047_0.jpg", + "im053_1.jpg", + "im057_1.jpg", + "im060_1.jpg", + "im063_1.jpg", + "im069_0.jpg", + "im074_0.jpg", + "im088_0.jpg", + "im095_0.jpg", + "im099_0.jpg", + "im101_0.jpg", + "im106_0.jpg" +], +``` + +  + +# Training +Now you are ready to train your model. Ensuring you are in the project root and the `hias-all-oneapi-classifier` conda environment is activated, run the following command: + +``` bash +python3 agent.py train +``` + +First the training and validation data will be prepared. At the end of this stage you will see the following output: + +``` bash +2021-09-18 21:49:44,700 - Agent - INFO - Data shape: (1584, 100, 100, 3) +2021-09-18 21:49:44,701 - Agent - INFO - Labels shape: (1584, 2) +2021-09-18 21:49:44,701 - Agent - INFO - Raw data: 792 +2021-09-18 21:49:44,701 - Agent - INFO - Raw negative data: 441 +2021-09-18 21:49:44,701 - Agent - INFO - Raw positive data: 351 +2021-09-18 21:49:44,701 - Agent - INFO - Augmented data: (1584, 100, 100, 3) +2021-09-18 21:49:44,701 - Agent - INFO - Labels: (1584, 2) +2021-09-18 21:49:44,923 - Agent - INFO - Training data: (1180, 100, 100, 3) +2021-09-18 21:49:44,923 - Agent - INFO - Training labels: (1180, 2) +2021-09-18 21:49:44,923 - Agent - INFO - Validation data: (404, 100, 100, 3) +2021-09-18 21:49:44,923 - Agent - INFO - Validation labels: (404, 2) +2021-09-18 21:49:44,925 - Agent - INFO - Data preperation complete. +``` + +Next you will see the model summary: + +``` bash +Model: "AllOneApiClassifier" +_________________________________________________________________ +Layer (type) Output Shape Param # +================================================================= +zero_padding2d (ZeroPadding2 (None, 104, 104, 3) 0 +_________________________________________________________________ +conv2d (Conv2D) (None, 100, 100, 30) 2280 +_________________________________________________________________ +zero_padding2d_1 (ZeroPaddin (None, 104, 104, 30) 0 +_________________________________________________________________ +conv2d_1 (Conv2D) (None, 100, 100, 30) 22530 +_________________________________________________________________ +max_pooling2d (MaxPooling2D) (None, 50, 50, 30) 0 +_________________________________________________________________ +flatten (Flatten) (None, 75000) 0 +_________________________________________________________________ +dense (Dense) (None, 2) 150002 +_________________________________________________________________ +activation (Activation) (None, 2) 0 +================================================================= +Total params: 174,812 +Trainable params: 174,812 +Non-trainable params: 0 +_________________________________________________________________ +``` + +And finally the model will begin training. + +  + +# Results +Below you will find the training results for 150 epochs. + +## Training Output + +``` bash +2021-09-19 00:02:28,030 - Agent - INFO - Metrics: loss 0.060633027169964104 +2021-09-19 00:02:28,030 - Agent - INFO - Metrics: acc 0.9727723 +2021-09-19 00:02:28,030 - Agent - INFO - Metrics: precision 0.9727723 +2021-09-19 00:02:28,030 - Agent - INFO - Metrics: recall 0.9727723 +2021-09-19 00:02:28,031 - Agent - INFO - Metrics: auc 0.9978372 + +2021-09-19 00:02:28,752 - Agent - INFO - Confusion Matrix: [[220 1] [ 10 173]] + +2021-09-19 00:02:28,861 - Agent - INFO - True Positives: 173(42.82178217821782%) +2021-09-19 00:02:28,861 - Agent - INFO - False Positives: 1(0.24752475247524752%) +2021-09-19 00:02:28,861 - Agent - INFO - True Negatives: 220(54.45544554455446%) +2021-09-19 00:02:28,861 - Agent - INFO - False Negatives: 10(2.4752475247524752%) +2021-09-19 00:02:28,861 - Agent - INFO - Specificity: 0.995475113122172 +2021-09-19 00:02:28,862 - Agent - INFO - Misclassification: 11(2.722772277227723%) +``` + +## Metrics Overview +| Accuracy | Recall | Precision | AUC/ROC | +| ---------- | ---------- | ---------- | ---------- | +| 0.9727723 | 0.9727723 | 0.9727723 | 0.9978372 | + +## ALL-IDB Required Metrics +| Figures of merit | Amount/Value | Percentage | +| -------------------- | ----- | ---------- | +| True Positives | 173 | 42.82178217821782% | +| False Positives | 1 | 0.24752475247524752% | +| True Negatives | 220 | 54.45544554455446% | +| False Negatives | 10 | 2.4752475247524752% | +| Misclassification | 11 | 2.722772277227723% | +| Sensitivity / Recall | 0.9727723 | 0.97% | +| Specificity | 0.995475113122172 | 100% | + +## Metrics Plots + +![Accuracy](../img/plots/accuracy.png) + +_Fig 1. Accuracy_ + +![Loss](../img/plots/loss.png) + +_Fig 2. Loss_ + +![Precision](../img/plots/precision.png) + +_Fig 3. Precision_ + +![Recall](../img/plots/recall.png) + +_Fig 4. Recall_ + +![AUC](../img/plots/auc.png) + +_Fig 5. AUC_ + +![Confusion Matrix](../img/plots/confusion-matrix.png) + +_Fig 6. Confusion Matrix_ + +  + +# Testing + +Now you will test the classifier on your development machine. You will use the 20 images that were removed from the training data in a previous part of this tutorial. + +To run the AI Agent in test mode use the following command: + +``` +python3 agent.py classify +``` + +You should see the following which shows you the network architecture: + +``` +Model: "AllOneApiClassifier" +_________________________________________________________________ +Layer (type) Output Shape Param # +================================================================= +zero_padding2d (ZeroPadding2 (None, 104, 104, 3) 0 +_________________________________________________________________ +conv2d (Conv2D) (None, 100, 100, 30) 2280 +_________________________________________________________________ +zero_padding2d_1 (ZeroPaddin (None, 104, 104, 30) 0 +_________________________________________________________________ +conv2d_1 (Conv2D) (None, 100, 100, 30) 22530 +_________________________________________________________________ +max_pooling2d (MaxPooling2D) (None, 50, 50, 30) 0 +_________________________________________________________________ +flatten (Flatten) (None, 75000) 0 +_________________________________________________________________ +dense (Dense) (None, 2) 150002 +_________________________________________________________________ +activation (Activation) (None, 2) 0 +================================================================= +Total params: 174,812 +Trainable params: 174,812 +Non-trainable params: 0 +``` + +Finally the application will start processing the test images and the results will be displayed in the console. + +``` bash +2021-09-19 00:03:34,593 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.15274596214294434 seconds. +2021-09-19 00:03:34,629 - Agent - INFO - Loaded test image model/data/test/Im028_1.jpg +2021-09-19 00:03:34,636 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.04251694679260254 seconds. +2021-09-19 00:03:34,713 - Agent - INFO - Loaded test image model/data/test/Im106_0.jpg +2021-09-19 00:03:34,720 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.0836799144744873 seconds. +2021-09-19 00:03:34,795 - Agent - INFO - Loaded test image model/data/test/Im101_0.jpg +2021-09-19 00:03:34,802 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08228707313537598 seconds. +2021-09-19 00:03:34,834 - Agent - INFO - Loaded test image model/data/test/Im024_1.jpg +2021-09-19 00:03:34,841 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.038298606872558594 seconds. +2021-09-19 00:03:34,914 - Agent - INFO - Loaded test image model/data/test/Im074_0.jpg +2021-09-19 00:03:34,921 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08018183708190918 seconds. +2021-09-19 00:03:34,996 - Agent - INFO - Loaded test image model/data/test/Im035_0.jpg +2021-09-19 00:03:35,003 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.0811767578125 seconds. +2021-09-19 00:03:35,034 - Agent - INFO - Loaded test image model/data/test/Im006_1.jpg +2021-09-19 00:03:35,041 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.03841996192932129 seconds. +2021-09-19 00:03:35,073 - Agent - INFO - Loaded test image model/data/test/Im020_1.jpg +2021-09-19 00:03:35,079 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.03810429573059082 seconds. +2021-09-19 00:03:35,147 - Agent - INFO - Loaded test image model/data/test/Im095_0.jpg +2021-09-19 00:03:35,154 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.07399249076843262 seconds. +2021-09-19 00:03:35,228 - Agent - INFO - Loaded test image model/data/test/Im069_0.jpg +2021-09-19 00:03:35,235 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08146858215332031 seconds. +2021-09-19 00:03:35,267 - Agent - INFO - Loaded test image model/data/test/Im031_1.jpg +2021-09-19 00:03:35,274 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.03837895393371582 seconds. +2021-09-19 00:03:35,348 - Agent - INFO - Loaded test image model/data/test/Im099_0.jpg +2021-09-19 00:03:35,355 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08129477500915527 seconds. +2021-09-19 00:03:35,387 - Agent - INFO - Loaded test image model/data/test/Im026_1.jpg +2021-09-19 00:03:35,394 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.03829550743103027 seconds. +2021-09-19 00:03:35,468 - Agent - INFO - Loaded test image model/data/test/Im057_1.jpg +2021-09-19 00:03:35,475 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.08077669143676758 seconds. +2021-09-19 00:03:35,550 - Agent - INFO - Loaded test image model/data/test/Im088_0.jpg +2021-09-19 00:03:35,557 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08171367645263672 seconds. +2021-09-19 00:03:35,632 - Agent - INFO - Loaded test image model/data/test/Im060_1.jpg +2021-09-19 00:03:35,638 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in 0.08168196678161621 seconds. +2021-09-19 00:03:35,713 - Agent - INFO - Loaded test image model/data/test/Im053_1.jpg +2021-09-19 00:03:35,720 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in 0.08159828186035156 seconds. +2021-09-19 00:03:35,794 - Agent - INFO - Loaded test image model/data/test/Im041_0.jpg +2021-09-19 00:03:35,801 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08086276054382324 seconds. +2021-09-19 00:03:35,876 - Agent - INFO - Loaded test image model/data/test/Im047_0.jpg +2021-09-19 00:03:35,883 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.08133721351623535 seconds. +2021-09-19 00:03:35,883 - Agent - INFO - Images Classified: 20 +2021-09-19 00:03:35,883 - Agent - INFO - True Positives: 8 +2021-09-19 00:03:35,883 - Agent - INFO - False Positives: 0 +2021-09-19 00:03:35,883 - Agent - INFO - True Negatives: 10 +2021-09-19 00:03:35,883 - Agent - INFO - False Negatives: 2 +2021-09-19 00:03:35,883 - Agent - INFO - Total Time Taken: 1.438812255859375 +``` + +In the current terminal, now use the following command: + +``` +python3 agent.py server +``` + +This will start the server on your training machine that exposes the model via a REST API. Now open a new terminal, navigate to the project root and use the following command: + +``` +python3 agent.py classify_http +``` + +This will start agent in HTTP Inference mode. The agent will loop through the testing data and send each image to the server for classification, the results are then displayed in the console. + +``` bash +2021-09-19 00:43:54,697 - Agent - INFO - Sending request for: model/data/test/Im063_1.jpg +2021-09-19 00:43:54,995 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.2976648807525635 seconds. +2021-09-19 00:43:54,995 - Agent - INFO - Sending request for: model/data/test/Im028_1.jpg +2021-09-19 00:43:55,110 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.11492085456848145 seconds. +2021-09-19 00:43:55,110 - Agent - INFO - Sending request for: model/data/test/Im106_0.jpg +2021-09-19 00:43:55,334 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.22387242317199707 seconds. +2021-09-19 00:43:55,334 - Agent - INFO - Sending request for: model/data/test/Im101_0.jpg +2021-09-19 00:43:55,557 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.22250127792358398 seconds. +2021-09-19 00:43:55,557 - Agent - INFO - Sending request for: model/data/test/Im024_1.jpg +2021-09-19 00:43:55,660 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.10355067253112793 seconds. +2021-09-19 00:43:55,660 - Agent - INFO - Sending request for: model/data/test/Im074_0.jpg +2021-09-19 00:43:55,879 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.21893024444580078 seconds. +2021-09-19 00:43:55,879 - Agent - INFO - Sending request for: model/data/test/Im035_0.jpg +2021-09-19 00:43:56,111 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.231856107711792 seconds. +2021-09-19 00:43:56,111 - Agent - INFO - Sending request for: model/data/test/Im006_1.jpg +2021-09-19 00:43:56,214 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.10265469551086426 seconds. +2021-09-19 00:43:56,214 - Agent - INFO - Sending request for: model/data/test/Im020_1.jpg +2021-09-19 00:43:56,315 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.1011204719543457 seconds. +2021-09-19 00:43:56,315 - Agent - INFO - Sending request for: model/data/test/Im095_0.jpg +2021-09-19 00:43:56,519 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.20377898216247559 seconds. +2021-09-19 00:43:56,519 - Agent - INFO - Sending request for: model/data/test/Im069_0.jpg +2021-09-19 00:43:56,742 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.22292089462280273 seconds. +2021-09-19 00:43:56,742 - Agent - INFO - Sending request for: model/data/test/Im031_1.jpg +2021-09-19 00:43:56,845 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.10294127464294434 seconds. +2021-09-19 00:43:56,845 - Agent - INFO - Sending request for: model/data/test/Im099_0.jpg +2021-09-19 00:43:57,067 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.2218623161315918 seconds. +2021-09-19 00:43:57,067 - Agent - INFO - Sending request for: model/data/test/Im026_1.jpg +2021-09-19 00:43:57,171 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.10353922843933105 seconds. +2021-09-19 00:43:57,171 - Agent - INFO - Sending request for: model/data/test/Im057_1.jpg +2021-09-19 00:43:57,394 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.2229921817779541 seconds. +2021-09-19 00:43:57,394 - Agent - INFO - Sending request for: model/data/test/Im088_0.jpg +2021-09-19 00:43:57,618 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.22388410568237305 seconds. +2021-09-19 00:43:57,618 - Agent - INFO - Sending request for: model/data/test/Im060_1.jpg +2021-09-19 00:43:57,841 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Negative) in 0.22273635864257812 seconds. +2021-09-19 00:43:57,841 - Agent - INFO - Sending request for: model/data/test/Im053_1.jpg +2021-09-19 00:43:58,066 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Negative) in 0.22502851486206055 seconds. +2021-09-19 00:43:58,066 - Agent - INFO - Sending request for: model/data/test/Im041_0.jpg +2021-09-19 00:43:58,290 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.2235252857208252 seconds. +2021-09-19 00:43:58,290 - Agent - INFO - Sending request for: model/data/test/Im047_0.jpg +2021-09-19 00:43:58,512 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.2222459316253662 seconds. +2021-09-19 00:43:58,512 - Agent - INFO - Images Classified: 20 +2021-09-19 00:43:58,512 - Agent - INFO - True Positives: 8 +2021-09-19 00:43:58,512 - Agent - INFO - False Positives: 0 +2021-09-19 00:43:58,513 - Agent - INFO - True Negatives: 10 +2021-09-19 00:43:58,513 - Agent - INFO - False Negatives: 2 +2021-09-19 00:43:58,513 - Agent - INFO - Total Time Taken: 3.8125267028808594 +``` + +# OpenVINO Intermediate Representation + +Finally you need to convert your frozen model to an Intermediate Representation. To do this, use the following command, replacing **YourProjectPath** with the path to your project home. + +``` +python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo_tf.py --input_model /YourProjectPath/hias-all-oneapi-classifier/model/freezing/frozen.pb --input_shape [1,100,100,3] --output_dir /YourProjectPath/hias-all-oneapi-classifier/model/ir --reverse_input_channels --generate_deprecated_IR_V7 +``` + +If all has been successful you should see the following output: + +``` bash +[ SUCCESS ] Generated IR version 10 model. +[ SUCCESS ] XML file: /YourProjectPath/hias-all-oneapi-classifier/model/ir/frozen.xml +[ SUCCESS ] BIN file: /YourProjectPath/hias-all-oneapi-classifier/model/ir/frozen.bin +[ SUCCESS ] Total execution time: 2.13 seconds. +[ SUCCESS ] Memory consumed: 245 MB. +``` + +  + +# Continue +Now you can continue with the Acute Lymphoblastic Leukemia oneAPI Classifier [Raspberry Pi installation guide](../installation/raspberry-pi.md) to set up your Rasperry Pi. + +  + +# Contributing + +The Peter Moss Acute Myeloid & Lymphoblastic Leukemia AI Research project encourages and youlcomes code contributions, bug fixes and enhancements from the Github. + +Please read the [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page. + +## Contributors + +- [Adam Milton-Barker](https://www.leukemiaresearchassociation.ai/team/adam-milton-barker "Adam Milton-Barker") - [Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain + +  + +# Versioning + +You use SemVer for versioning. For the versions available, see [Releases](../../releases "Releases"). + +  + +# License + +This project is licensed under the **MIT License** - see the [LICENSE](../../LICENSE "LICENSE") file for details. + +  + +# Bugs/Issues + +You use the [repo issues](../../issues "repo issues") to track bugs and general requests related to using this project. See [CONTRIBUTING](../../CONTRIBUTING.md "CONTRIBUTING") for more info on how to submit bugs, feature requests and proposals. \ No newline at end of file diff --git a/documentation/getting-started.md b/documentation/getting-started.md deleted file mode 100644 index 3fdc59f..0000000 --- a/documentation/getting-started.md +++ /dev/null @@ -1,788 +0,0 @@ -# Asociación de Investigacion en Inteligencia Artificial Para la Leucemia Peter Moss -## Acute Lymphoblastic Leukemia oneAPI Classifier -### Getting Started - -![Acute Lymphoblastic Leukemia oneAPI Classifier](../assets/images/all-oneapi-classifier-2020.png) - -  - -# Table Of Contents - -- [Introduction](#introduction) - - [Network Architecture](#network-architecture) -- [Installation](#installation) -- [Data](#data) - - [Data Augmentation](#data-augmentation) - - [Application Testing Data](#application-testing-data) -- [Configuration](#configuration) -- [Training](#training) - - [Metrics](#metrics) - - [Start Training](#start-training) - - [Training Data](#training-data) - - [Model Summary](#model-summary) - - [Training Results](#training-results) - - [Metrics Overview](#metrics-overview) - - [ALL-IDB Required Metrics](#all-idb-required-metrics) -- [Testing](#testing) -- [OpenVINO Intermediate Representation](#openvino-intermediate-representation) -- [Raspberry Pi 4](#raspberry-pi-4) - - [Raspberry Pi OS](#raspberry-pi-os) - - [Intel® Distribution of OpenVINO™ Toolkit](#intel-distribution-of-openvino-toolkit) - - [Intel® Movidius™ Neural Compute Stick 2](#intel-movidius-neural-compute-stick-2) - - [Software Install](#software-install) - - [Local Testing](#local-testing) - - [Server Testing](#server-testing) -- [HIAS UI](#hias-ui) - - [NGINX](#nginx) - - [Inference](#inference) - - [Verification](#verification) -- [Contributing](#contributing) - - [Contributors](#contributors) -- [Versioning](#versioning) -- [License](#license) -- [Bugs/Issues](#bugs-issues) - -  - -# Introduction -This guide will guide you through the installation process for the Acute Lymphoblastic Leukemia oneAPI Classifier. - -## Network Architecture -You will build a Convolutional Neural Network based on the proposed architecture in [Acute Leukemia Classification Using Convolution Neural Network In Clinical Decision Support System](https://airccj.org/CSCP/vol7/csit77505.pdf "Acute Leukemia Classification Using Convolution Neural Network In Clinical Decision Support System"). The network will consist of the following 5 layers (missing out the zero padding layers). Note you are usng an conv sizes of (100x100x30) whereas in the paper, the authors use (50x50x30). - -- Conv layer (100x100x30) -- Conv layer (100x100x30) -- Max-Pooling layer (50x50x30) -- Fully Connected layer (2 neurons) -- Softmax layer (Output 2) - -  - -# Installation -First you need to install the required software for training the model. Below are the available installation guides: - -- [Ubuntu installation guide](installation/ubuntu.md) (Training). -- [Raspberry Pi 4 installation guide](installation/rpi4.md) (Inference on the edge). - -  - -# Data -You need to be granted access to use the Acute Lymphoblastic Leukemia Image Database for Image Processing dataset. You can find the application form and information about getting access to the dataset on [this page](https://homes.di.unimi.it/scotti/all/#download) as well as information on how to contribute back to the project [here](https://homes.di.unimi.it/scotti/all/results.php). - -_If you are not able to obtain a copy of the dataset please feel free to try this tutorial on your own dataset._ - -Once you have your data you need to add it to the project filesystem. You will notice the data folder in the Model directory, **model/data**, inside you have **train** & **test**. Add all of the images from the ALL_IDB1 dataset to the **model/data/train** folder. - -## Data Augmentation - -You will create an augmented dataset based on the [Leukemia Blood Cell Image Classification Using Convolutional Neural Network](http://www.ijcte.org/vol10/1198-H0012.pdf "Leukemia Blood Cell Image Classification Using Convolutional Neural Network") by T. T. P. Thanh, Caleb Vununu, Sukhrob Atoev, Suk-Hwan Lee, and Ki-Ryong Kwon. In this case, you will use more rotated images to increase the dataset further. - -## Application testing data - -In the data processing stage, ten negative images and ten positive images are removed from the dataset and moved to the **model/data/test/** directory. This data is not seen by the network during the training process, and is used by applications. - -To ensure your model gets the same results, please use the same test images. By default HIAS compatible projects will be expecting the same test images. You can also try with your own image selection, however results may vary and you will need to make additional changes to our HIAS compatible projects. - -To specify which test images to use modify the [configuration/config.json](../configuration/config.json) file as shown below: - -``` -"test_data": [ - "im006_1.jpg", - "im020_1.jpg", - "im024_1.jpg", - "im026_1.jpg", - "im028_1.jpg", - "im031_1.jpg", - "im035_0.jpg", - "im041_0.jpg", - "im047_0.jpg", - "im053_1.jpg", - "im057_1.jpg", - "im060_1.jpg", - "im063_1.jpg", - "im069_0.jpg", - "im074_0.jpg", - "im088_0.jpg", - "im095_0.jpg", - "im099_0.jpg", - "im101_0.jpg", - "im106_0.jpg" -], -``` - -  - -# Configuration -[configuration/config.json](../configuration/config.json "configuration/config.json") holds the configuration for our application. - -- Change **agent->cores** to the number of cores you have on your training computer. -- Change **agent->server** to the local IP of your training device. -- Change **agent->port** to a different number. - -
View file contents -

- -``` -{ - "agent": { - "cores": 8, - "server": "", - "port": 1234, - "params": [ - "train", - "classify", - "server", - "classify_http" - ] - }, - "data": { - "dim": 100, - "file_type": ".jpg", - "labels": [0, 1], - "rotations": 10, - "seed": 2, - "split": 0.3, - "test": "model/data/test", - "test_data": [ - "Im006_1.jpg", - "Im020_1.jpg", - "Im024_1.jpg", - "Im026_1.jpg", - "Im028_1.jpg", - "Im031_1.jpg", - "Im035_0.jpg", - "Im041_0.jpg", - "Im047_0.jpg", - "Im053_1.jpg", - "Im057_1.jpg", - "Im060_1.jpg", - "Im063_1.jpg", - "Im069_0.jpg", - "Im074_0.jpg", - "Im088_0.jpg", - "Im095_0.jpg", - "Im099_0.jpg", - "Im101_0.jpg", - "Im106_0.jpg" - ], - "train_dir": "model/data/train", - "valid_types": [ - ".JPG", - ".JPEG", - ".PNG", - ".GIF", - ".jpg", - ".jpeg", - ".png", - ".gif" - ] - }, - "model": { - "device": "CPU", - "freezing_log_dir": "model/freezing", - "frozen": "frozen.pb", - "ir": "model/ir/frozen.xml", - "model": "model/model.json", - "saved_model_dir": "model", - "weights": "model/weights.h5" - }, - "train": { - "batch": 100, - "decay_adam": 1e-6, - "epochs": 150, - "learning_rate_adam": 1e-4, - "val_steps": 10 - } -} -``` - -

-

- -The configuration object contains 4 Json Objects (agent, data, model and train). Agent has the information used to set up your server, data has the configuration related to preparing the training and validation data, model holds the model file paths, and train holds the training parameters. - -  - -# Training -Now you are ready to train your model. - -## Metrics -We can use metrics to measure the effectiveness of our model. In this network you will use the following metrics: - -``` -tf.keras.metrics.BinaryAccuracy(name='accuracy'), -tf.keras.metrics.Precision(name='precision'), -tf.keras.metrics.Recall(name='recall'), -tf.keras.metrics.AUC(name='auc') -``` - -These metrics will be displayed and plotted once our model is trained. A useful tutorial while working on the metrics was the [Classification on imbalanced data](https://www.tensorflow.org/tutorials/structured_data/imbalanced_data) tutorial on Tensorflow's website. - -## Start Training -Ensuring you have completed all previous steps, you can start training using the following command. - -``` -python agent.py train -``` - -This tells the application to start training the model. - -## Training Data -First the training and validation data will be prepared. - -``` -2021-05-02 18: 43: 57, 164 - Agent - INFO - Data shape: (1584, 100, 100, 3) -2021-05-02 18: 43: 57, 165 - Agent - INFO - Labels shape: (1584, 2) -2021-05-02 18: 43: 57, 165 - Agent - INFO - Raw data: 792 -2021-05-02 18: 43: 57, 165 - Agent - INFO - Raw negative data: 441 -2021-05-02 18: 43: 57, 166 - Agent - INFO - Raw positive data: 351 -2021-05-02 18: 43: 57, 166 - Agent - INFO - Augmented data: (1584, 100, 100, 3) -2021-05-02 18: 43: 57, 166 - Agent - INFO - Labels: (1584, 2) -2021-05-02 18: 43: 57, 334 - Agent - INFO - Training data: (1180, 100, 100, 3) -2021-05-02 18: 43: 57, 334 - Agent - INFO - Training labels: (1180, 2) -2021-05-02 18: 43: 57, 334 - Agent - INFO - Validation data: (404, 100, 100, 3) -2021-05-02 18: 43: 57, 334 - Agent - INFO - Validation labels: (404, 2) -2021-05-02 18: 43: 57, 359 - Agent - INFO - Data preperation complete. -``` - -### Model Summary - -Before the model begins training, you will be shown the model summary. - -``` -Model: "AllOneApiClassifier" -_________________________________________________________________ -Layer (type) Output Shape Param # -================================================================= -zero_padding2d (ZeroPadding2 (None, 104, 104, 3) 0 -_________________________________________________________________ -conv2d (Conv2D) (None, 100, 100, 30) 2280 -_________________________________________________________________ -zero_padding2d_1 (ZeroPaddin (None, 104, 104, 30) 0 -_________________________________________________________________ -conv2d_1 (Conv2D) (None, 100, 100, 30) 22530 -_________________________________________________________________ -max_pooling2d (MaxPooling2D) (None, 50, 50, 30) 0 -_________________________________________________________________ -flatten (Flatten) (None, 75000) 0 -_________________________________________________________________ -dense (Dense) (None, 2) 150002 -_________________________________________________________________ -activation (Activation) (None, 2) 0 -================================================================= -Total params: 174,812 -Trainable params: 174,812 -Non-trainable params: 0 -_________________________________________________________________ -2021-05-02 18:43:57,414 - Agent - INFO - Network initialization complete. -2021-05-02 18:43:57,414 - Agent - INFO - Using Adam Optimizer. -Train on 1180 samples, validate on 404 samples -``` - -Our network matches the architecture proposed in the paper. - -## Training Results -Below are the training results for 150 epochs. - -Adam Optimizer Results - -_Fig 2. Accuracy_ - -Loss - -_Fig 3. Loss_ - -Precision - -_Fig 4. Precision_ - -Recall - -_Fig 5. Recall_ - -AUC - -_Fig 6. AUC_ - -AUC - -_Fig 7. Confusion Matrix_ - -``` -2021-05-02 14:33:48,514 - Agent - INFO - Metrics: loss 0.04992164568145677 -2021-05-02 14:33:48,514 - Agent - INFO - Metrics: acc 0.9826733 -2021-05-02 14:33:48,514 - Agent - INFO - Metrics: precision 0.9826733 -2021-05-02 14:33:48,515 - Agent - INFO - Metrics: recall 0.9826733 -2021-05-02 14:33:48,515 - Agent - INFO - Metrics: auc 0.99847436 - -2021-05-02 14:33:49,146 - Agent - INFO - Confusion Matrix: [[220 1] [ 6 177]] - -2021-05-02 14:33:49,278 - Agent - INFO - True Positives: 177(43.81188118811881%) -2021-05-02 14:33:49,278 - Agent - INFO - False Positives: 1(0.24752475247524752%) -2021-05-02 14:33:49,278 - Agent - INFO - True Negatives: 220(54.45544554455446%) -2021-05-02 14:33:49,278 - Agent - INFO - False Negatives: 6(1.4851485148514851%) -2021-05-02 14:33:49,278 - Agent - INFO - Specificity: 0.995475113122172 -2021-05-02 14:33:49,278 - Agent - INFO - Misclassification: 7(1.7326732673267327%) -``` - -## Metrics Overview -| Accuracy | Recall | Precision | AUC/ROC | -| ---------- | ---------- | ---------- | ---------- | -| 0.9826733 | 0.9826733 | 0.9826733 | 0.99847436 | - - -## ALL-IDB Required Metrics -| Figures of merit | Amount/Value | Percentage | -| -------------------- | ----- | ---------- | -| True Positives | 177 | 43.81188118811881% | -| False Positives | 1 | 0.24752475247524752% | -| True Negatives | 220 | 54.45544554455446% | -| False Negatives | 6 | 1.4851485148514851% | -| Misclassification | 7 | 1.7326732673267327% | -| Sensitivity / Recall | 0.9826733 | 0.98% | -| Specificity | 0.995475113122172 | 100% | - -  - -# Testing - -Now you will test the classifier on your training machine. You will use the 20 images that were removed from the training data in a previous part of this tutorial. - -To run the AI Agent in test mode use the following command: - -``` -python3 agenty.py classify -``` - -You should see the following which shows you the network architecture: - -``` -Model: "AllOneApiClassifier" -_________________________________________________________________ -Layer (type) Output Shape Param # -================================================================= -zero_padding2d (ZeroPadding2 (None, 104, 104, 3) 0 -_________________________________________________________________ -conv2d (Conv2D) (None, 100, 100, 30) 2280 -_________________________________________________________________ -zero_padding2d_1 (ZeroPaddin (None, 104, 104, 30) 0 -_________________________________________________________________ -conv2d_1 (Conv2D) (None, 100, 100, 30) 22530 -_________________________________________________________________ -max_pooling2d (MaxPooling2D) (None, 50, 50, 30) 0 -_________________________________________________________________ -flatten (Flatten) (None, 75000) 0 -_________________________________________________________________ -dense (Dense) (None, 2) 150002 -_________________________________________________________________ -activation (Activation) (None, 2) 0 -================================================================= -Total params: 174,812 -Trainable params: 174,812 -Non-trainable params: 0 -``` - -Finally the application will start processing the test images and the results will be displayed in the console. - -
View output -

- -``` -2021-05-03 22:56:09,704 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.1585226058959961 seconds. -2021-05-03 22:56:09,739 - Agent - INFO - Loaded test image model/data/test/Im028_1.jpg -2021-05-03 22:56:09,760 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.05670356750488281 seconds. -2021-05-03 22:56:09,845 - Agent - INFO - Loaded test image model/data/test/Im106_0.jpg -2021-05-03 22:56:09,867 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10596728324890137 seconds. -2021-05-03 22:56:09,949 - Agent - INFO - Loaded test image model/data/test/Im101_0.jpg -2021-05-03 22:56:09,971 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10411334037780762 seconds. -2021-05-03 22:56:10,007 - Agent - INFO - Loaded test image model/data/test/Im024_1.jpg -2021-05-03 22:56:10,032 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.06063079833984375 seconds. -2021-05-03 22:56:10,115 - Agent - INFO - Loaded test image model/data/test/Im074_0.jpg -2021-05-03 22:56:10,137 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10519862174987793 seconds. -2021-05-03 22:56:10,220 - Agent - INFO - Loaded test image model/data/test/Im035_0.jpg -2021-05-03 22:56:10,244 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10655450820922852 seconds. -2021-05-03 22:56:10,277 - Agent - INFO - Loaded test image model/data/test/Im006_1.jpg -2021-05-03 22:56:10,298 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.05443882942199707 seconds. -2021-05-03 22:56:10,332 - Agent - INFO - Loaded test image model/data/test/Im020_1.jpg -2021-05-03 22:56:10,353 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.054483890533447266 seconds. -2021-05-03 22:56:10,424 - Agent - INFO - Loaded test image model/data/test/Im095_0.jpg -2021-05-03 22:56:10,446 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 0.0931251049041748 seconds. -2021-05-03 22:56:10,529 - Agent - INFO - Loaded test image model/data/test/Im069_0.jpg -2021-05-03 22:56:10,551 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10473918914794922 seconds. -2021-05-03 22:56:10,588 - Agent - INFO - Loaded test image model/data/test/Im031_1.jpg -2021-05-03 22:56:10,610 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.05825352668762207 seconds. -2021-05-03 22:56:10,693 - Agent - INFO - Loaded test image model/data/test/Im099_0.jpg -2021-05-03 22:56:10,715 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10500550270080566 seconds. -2021-05-03 22:56:10,748 - Agent - INFO - Loaded test image model/data/test/Im026_1.jpg -2021-05-03 22:56:10,776 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.06057333946228027 seconds. -2021-05-03 22:56:10,858 - Agent - INFO - Loaded test image model/data/test/Im057_1.jpg -2021-05-03 22:56:10,880 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.10422873497009277 seconds. -2021-05-03 22:56:10,963 - Agent - INFO - Loaded test image model/data/test/Im088_0.jpg -2021-05-03 22:56:10,985 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10511159896850586 seconds. -2021-05-03 22:56:11,067 - Agent - INFO - Loaded test image model/data/test/Im060_1.jpg -2021-05-03 22:56:11,092 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.10669517517089844 seconds. -2021-05-03 22:56:11,177 - Agent - INFO - Loaded test image model/data/test/Im053_1.jpg -2021-05-03 22:56:11,199 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in 0.10645747184753418 seconds. -2021-05-03 22:56:11,282 - Agent - INFO - Loaded test image model/data/test/Im041_0.jpg -2021-05-03 22:56:11,303 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10442733764648438 seconds. -2021-05-03 22:56:11,387 - Agent - INFO - Loaded test image model/data/test/Im047_0.jpg -2021-05-03 22:56:11,408 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.10486173629760742 seconds. -2021-05-03 22:56:11,409 - Agent - INFO - Images Classified: 20 -2021-05-03 22:56:11,409 - Agent - INFO - True Positives: 9 -2021-05-03 22:56:11,409 - Agent - INFO - False Positives: 1 -2021-05-03 22:56:11,409 - Agent - INFO - True Negatives: 9 -2021-05-03 22:56:11,409 - Agent - INFO - False Negatives: 1 -2021-05-03 22:56:11,409 - Agent - INFO - Total Time Taken: 1.8600921630859375 -``` -

-

- -In the current terminal, now use the following command: - -``` -python3 agenty.py server -``` - -This will start the server on your training machine that exposes the model via a REST API. Now open a new terminal, navigate to the project root and use the following command: - -``` -python3 agenty.py classify_http -``` - -This will start agent in HTTP Inference mode. The agent will loop through the testing data and send each image to the server for classification, the results are then displayed in the console. - -
View output -

``` -2021-05-03 23:04:49,099 - Agent - INFO - Sending request for: model/data/test/Im063_1.jpg -2021-05-03 23:04:49,594 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.4954085350036621 seconds. -2021-05-03 23:04:49,594 - Agent - INFO - Sending request for: model/data/test/Im028_1.jpg -2021-05-03 23:04:49,826 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.2316751480102539 seconds. -2021-05-03 23:04:49,826 - Agent - INFO - Sending request for: model/data/test/Im106_0.jpg -2021-05-03 23:04:50,240 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.4138331413269043 seconds. -2021-05-03 23:04:50,240 - Agent - INFO - Sending request for: model/data/test/Im101_0.jpg -2021-05-03 23:04:50,656 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.41562962532043457 seconds. -2021-05-03 23:04:50,656 - Agent - INFO - Sending request for: model/data/test/Im024_1.jpg -2021-05-03 23:04:50,854 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.19834423065185547 seconds. -2021-05-03 23:04:50,854 - Agent - INFO - Sending request for: model/data/test/Im074_0.jpg -2021-05-03 23:04:51,271 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.4169602394104004 seconds. -2021-05-03 23:04:51,271 - Agent - INFO - Sending request for: model/data/test/Im035_0.jpg -2021-05-03 23:04:51,697 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.42549872398376465 seconds. -2021-05-03 23:04:51,697 - Agent - INFO - Sending request for: model/data/test/Im006_1.jpg -2021-05-03 23:04:51,901 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.20358586311340332 seconds. -2021-05-03 23:04:51,901 - Agent - INFO - Sending request for: model/data/test/Im020_1.jpg -2021-05-03 23:04:52,099 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.1985161304473877 seconds. -2021-05-03 23:04:52,099 - Agent - INFO - Sending request for: model/data/test/Im095_0.jpg -2021-05-03 23:04:52,476 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 0.3768906593322754 seconds. -2021-05-03 23:04:52,476 - Agent - INFO - Sending request for: model/data/test/Im069_0.jpg -2021-05-03 23:04:52,883 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.4070127010345459 seconds. -2021-05-03 23:04:52,884 - Agent - INFO - Sending request for: model/data/test/Im031_1.jpg -2021-05-03 23:04:53,084 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.20087146759033203 seconds. -2021-05-03 23:04:53,085 - Agent - INFO - Sending request for: model/data/test/Im099_0.jpg -2021-05-03 23:04:53,498 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 0.4138002395629883 seconds. -2021-05-03 23:04:53,498 - Agent - INFO - Sending request for: model/data/test/Im026_1.jpg -2021-05-03 23:04:53,700 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.20121002197265625 seconds. -2021-05-03 23:04:53,700 - Agent - INFO - Sending request for: model/data/test/Im057_1.jpg -2021-05-03 23:04:54,108 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.40814685821533203 seconds. -2021-05-03 23:04:54,108 - Agent - INFO - Sending request for: model/data/test/Im088_0.jpg -2021-05-03 23:04:54,519 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.41127872467041016 seconds. -2021-05-03 23:04:54,519 - Agent - INFO - Sending request for: model/data/test/Im060_1.jpg -2021-05-03 23:04:54,938 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Negative) in 0.41834306716918945 seconds. -2021-05-03 23:04:54,938 - Agent - INFO - Sending request for: model/data/test/Im053_1.jpg -2021-05-03 23:04:55,350 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.41184020042419434 seconds. -2021-05-03 23:04:55,350 - Agent - INFO - Sending request for: model/data/test/Im041_0.jpg -2021-05-03 23:04:55,759 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.4095749855041504 seconds. -2021-05-03 23:04:55,760 - Agent - INFO - Sending request for: model/data/test/Im047_0.jpg -2021-05-03 23:04:56,173 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 0.41382861137390137 seconds. -2021-05-03 23:04:56,173 - Agent - INFO - Images Classified: 20 -2021-05-03 23:04:56,174 - Agent - INFO - True Positives: 9 -2021-05-03 23:04:56,174 - Agent - INFO - False Positives: 2 -2021-05-03 23:04:56,174 - Agent - INFO - True Negatives: 8 -2021-05-03 23:04:56,174 - Agent - INFO - False Negatives: 1 -2021-05-03 23:04:56,174 - Agent - INFO - Total Time Taken: 7.072249174118042 -``` -

-

- -# OpenVINO Intermediate Representation - -Now you need to convert your frozen model to an Intermediate Representation. To do this, use the following command, replacing **YourProjectPath** with the path to your project home. - -``` -python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo_tf.py --input_model /YourProjectPath/ALL-Classifier-2020/model/freezing/frozen.pb --input_shape [1,100,100,3] --output_dir /YourProjectPath/ALL-Classifier-2020/model/ir --reverse_input_channels --generate_deprecated_IR_V7 -``` - -  - -# Raspberry Pi 4 - -The final part of this project is to set up the Acute Lymphoblastic Leukemia oneAPI Classifier on a Raspberry Pi 4 and install OpenVINO. - -## Raspberry Pi OS -For this Project, the operating system choice is [Raspberry Pi OS](https://www.raspberrypi.org/downloads/raspberry-pi-os/ "Raspberry Pi OS"). - -## Intel® Distribution of OpenVINO™ Toolkit -To install Intel® Distribution of OpenVINO™ Toolkit for Raspberry Pi, navigate to the home directory on your Raspberry Pi and use the following commands: - -``` - wget https://download.01.org/opencv/2021/openvinotoolkit/2021.2/l_openvino_toolkit_runtime_raspbian_p_2021.2.185.tgz -``` -``` - sudo mkdir -p /opt/intel/openvino - sudo tar -xf l_openvino_toolkit_runtime_raspbian_p_2021.2.185.tgz --strip 1 -C /opt/intel/openvino -``` -``` - sudo apt install cmake - source /opt/intel/openvino/bin/setupvars.sh - echo "source /opt/intel/openvino/bin/setupvars.sh" >> ~/.bashrc -``` - -## Intel® Movidius™ Neural Compute Stick 2 -Now we will set up ready for Neural Compute Stick 2. -``` - sudo usermod -a -G users "$(whoami)" -``` -Now close your existing terminal and open a new open. Once in your new terminal use the following commands: -``` - sh /opt/intel/openvino/install_dependencies/install_NCS_udev_rules.sh -``` - -## Transfer Files - -Next you need to transfer the project folder to your Raspberry Pi, make sure that you have all of the files from the model directory. - -You need to make some minor changes to the code. - -In **agent.py** comment out line 43 and uncomment line 44. - -``` -43 #from modules.model import model -44 from modules.model_openvino import model_openvino -``` - -In **modules/AbstractAgent.py** do the same changes to lines 38 and 39. - -``` -38 #from modules.model import model -39 from modules.model_openvino import model_openvino -``` - -In **configuration/credentials.json** change the IP of the server (server->ip) to match the IP of your Raspberry Pi and (server->port) to match the port you want to run on, remember to make sure the port is open. - -## Software Install - -All other requirements are included in **scripts/install_rpi4.sh**. You can run this file on machine by navigating to the project root in terminal and using the commands below: - -``` - sed -i 's/\r//' scripts/install_rpi4.sh - sh scripts/install_rpi4.sh -``` - -## Local Testing - -Now you will test the classifier on your Raspberry Pi 4 using OpenVINO & Neural Compute Stick 2. You will use the 20 images that were removed from the training data in a previous part of this tutorial. - -To run the AI Agent in test mode use the following command: - -``` -python3 agenty.py classify_openvino -``` - -You should see the application will start processing the test images and the results will be displayed in the console. - -
View output -

-``` -2021-05-03 23:43:22,310 - Agent - INFO - OpenVINO loaded. -2021-05-03 23:43:22,633 - Agent - INFO - Loaded test image model/data/test/Im099_0.jpg -2021-05-03 23:43:24,387 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 2.07651948928833 seconds. -2021-05-03 23:43:24,711 - Agent - INFO - Loaded test image model/data/test/Im047_0.jpg -2021-05-03 23:43:24,727 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.3396339416503906 seconds. -2021-05-03 23:43:25,050 - Agent - INFO - Loaded test image model/data/test/Im106_0.jpg -2021-05-03 23:43:25,066 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.33814501762390137 seconds. -2021-05-03 23:43:25,206 - Agent - INFO - Loaded test image model/data/test/Im020_1.jpg -2021-05-03 23:43:25,221 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.15483784675598145 seconds. -2021-05-03 23:43:25,518 - Agent - INFO - Loaded test image model/data/test/Im095_0.jpg -2021-05-03 23:43:25,533 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 0.31178736686706543 seconds. -2021-05-03 23:43:25,858 - Agent - INFO - Loaded test image model/data/test/Im053_1.jpg -2021-05-03 23:43:25,874 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in 0.33997583389282227 seconds. -2021-05-03 23:43:26,013 - Agent - INFO - Loaded test image model/data/test/Im024_1.jpg -2021-05-03 23:43:26,029 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.15473532676696777 seconds. -2021-05-03 23:43:26,351 - Agent - INFO - Loaded test image model/data/test/Im069_0.jpg -2021-05-03 23:43:26,367 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.33750343322753906 seconds. -2021-05-03 23:43:26,687 - Agent - INFO - Loaded test image model/data/test/Im074_0.jpg -2021-05-03 23:43:26,703 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.3357832431793213 seconds. -2021-05-03 23:43:26,843 - Agent - INFO - Loaded test image model/data/test/Im006_1.jpg -2021-05-03 23:43:26,858 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.1549975872039795 seconds. -2021-05-03 23:43:27,183 - Agent - INFO - Loaded test image model/data/test/Im088_0.jpg -2021-05-03 23:43:27,198 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.33968353271484375 seconds. -2021-05-03 23:43:27,521 - Agent - INFO - Loaded test image model/data/test/Im060_1.jpg -2021-05-03 23:43:27,536 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.33792710304260254 seconds. -2021-05-03 23:43:27,860 - Agent - INFO - Loaded test image model/data/test/Im101_0.jpg -2021-05-03 23:43:27,875 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.33822011947631836 seconds. -2021-05-03 23:43:28,015 - Agent - INFO - Loaded test image model/data/test/Im031_1.jpg -2021-05-03 23:43:28,030 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.15496253967285156 seconds. -2021-05-03 23:43:28,353 - Agent - INFO - Loaded test image model/data/test/Im063_1.jpg -2021-05-03 23:43:28,368 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.3369777202606201 seconds. -2021-05-03 23:43:28,692 - Agent - INFO - Loaded test image model/data/test/Im035_0.jpg -2021-05-03 23:43:28,707 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.33898305892944336 seconds. -2021-05-03 23:43:29,034 - Agent - INFO - Loaded test image model/data/test/Im057_1.jpg -2021-05-03 23:43:29,049 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.3415520191192627 seconds. -2021-05-03 23:43:29,372 - Agent - INFO - Loaded test image model/data/test/Im041_0.jpg -2021-05-03 23:43:29,388 - Agent - INFO - Acute Lymphoblastic Leukemia correctly not detected (True Negative) in 0.33808064460754395 seconds. -2021-05-03 23:43:29,528 - Agent - INFO - Loaded test image model/data/test/Im028_1.jpg -2021-05-03 23:43:29,543 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.15532946586608887 seconds. -2021-05-03 23:43:29,683 - Agent - INFO - Loaded test image model/data/test/Im026_1.jpg -2021-05-03 23:43:29,698 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.15422916412353516 seconds. -2021-05-03 23:43:29,698 - Agent - INFO - Images Classifier: 20 -2021-05-03 23:43:29,699 - Agent - INFO - True Positives: 9 -2021-05-03 23:43:29,699 - Agent - INFO - False Positives: 1 -2021-05-03 23:43:29,699 - Agent - INFO - True Negatives: 9 -2021-05-03 23:43:29,699 - Agent - INFO - False Negatives: 1 -2021-05-03 23:43:29,699 - Agent - INFO - Total Time Taken: 7.379864454269409 -``` -

-

- - -## Server Testing - -In the current terminal, now use the following command: - -``` -python3 agenty.py server_openvino -``` - -This will start the server on your training machine that exposes the model via a REST API. Now open a new terminal, navigate to the project root and use the following command: - -``` -python3 agenty.py classify_openvino_http -``` - -This will start agent in HTTP Inference mode. The agent will loop through the testing data and send each image to the server for classification, the results are then displayed in the console. - -
View output -

-``` -2021-05-03 23:50:26,485 - Agent - INFO - OpenVINO class initialization complete. -2021-05-03 23:50:26,485 - Agent - INFO - Sending request for: model/data/test/Im099_0.jpg -2021-05-03 23:50:27,847 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3621375560760498 seconds. -2021-05-03 23:50:27,848 - Agent - INFO - Sending request for: model/data/test/Im047_0.jpg -2021-05-03 23:50:29,204 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3563497066497803 seconds. -2021-05-03 23:50:29,204 - Agent - INFO - Sending request for: model/data/test/Im106_0.jpg -2021-05-03 23:50:30,561 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3568847179412842 seconds. -2021-05-03 23:50:30,562 - Agent - INFO - Sending request for: model/data/test/Im020_1.jpg -2021-05-03 23:50:31,233 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6714181900024414 seconds. -2021-05-03 23:50:31,234 - Agent - INFO - Sending request for: model/data/test/Im095_0.jpg -2021-05-03 23:50:32,491 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in 1.2567052841186523 seconds. -2021-05-03 23:50:32,491 - Agent - INFO - Sending request for: model/data/test/Im053_1.jpg -2021-05-03 23:50:33,885 - Agent - INFO - Acute Lymphoblastic Leukemia incorrectly detected (False Negative) in 1.3938932418823242 seconds. -2021-05-03 23:50:33,885 - Agent - INFO - Sending request for: model/data/test/Im024_1.jpg -2021-05-03 23:50:34,528 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6430263519287109 seconds. -2021-05-03 23:50:34,529 - Agent - INFO - Sending request for: model/data/test/Im069_0.jpg -2021-05-03 23:50:35,881 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3517706394195557 seconds. -2021-05-03 23:50:35,881 - Agent - INFO - Sending request for: model/data/test/Im074_0.jpg -2021-05-03 23:50:37,229 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3475196361541748 seconds. -2021-05-03 23:50:37,229 - Agent - INFO - Sending request for: model/data/test/Im006_1.jpg -2021-05-03 23:50:37,881 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6519880294799805 seconds. -2021-05-03 23:50:37,881 - Agent - INFO - Sending request for: model/data/test/Im088_0.jpg -2021-05-03 23:50:39,239 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3579890727996826 seconds. -2021-05-03 23:50:39,240 - Agent - INFO - Sending request for: model/data/test/Im060_1.jpg -2021-05-03 23:50:40,595 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 1.3549017906188965 seconds. -2021-05-03 23:50:40,595 - Agent - INFO - Sending request for: model/data/test/Im101_0.jpg -2021-05-03 23:50:41,952 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3565378189086914 seconds. -2021-05-03 23:50:41,952 - Agent - INFO - Sending request for: model/data/test/Im031_1.jpg -2021-05-03 23:50:42,634 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6819820404052734 seconds. -2021-05-03 23:50:42,635 - Agent - INFO - Sending request for: model/data/test/Im063_1.jpg -2021-05-03 23:50:43,989 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 1.3541193008422852 seconds. -2021-05-03 23:50:43,989 - Agent - INFO - Sending request for: model/data/test/Im035_0.jpg -2021-05-03 23:50:45,382 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3929808139801025 seconds. -2021-05-03 23:50:45,383 - Agent - INFO - Sending request for: model/data/test/Im057_1.jpg -2021-05-03 23:50:46,735 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 1.3522601127624512 seconds. -2021-05-03 23:50:46,735 - Agent - INFO - Sending request for: model/data/test/Im041_0.jpg -2021-05-03 23:50:48,088 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Negative) in 1.3523664474487305 seconds. -2021-05-03 23:50:48,088 - Agent - INFO - Sending request for: model/data/test/Im028_1.jpg -2021-05-03 23:50:48,735 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6467878818511963 seconds. -2021-05-03 23:50:48,735 - Agent - INFO - Sending request for: model/data/test/Im026_1.jpg -2021-05-03 23:50:49,385 - Agent - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive) in 0.6494393348693848 seconds. -2021-05-03 23:50:49,385 - Agent - INFO - Images Classified: 20 -2021-05-03 23:50:49,385 - Agent - INFO - True Positives: 9 -2021-05-03 23:50:49,386 - Agent - INFO - False Positives: 1 -2021-05-03 23:50:49,386 - Agent - INFO - True Negatives: 9 -2021-05-03 23:50:49,386 - Agent - INFO - False Negatives: 1 -2021-05-03 23:50:49,386 - Agent - INFO - Total Time Taken: 22.89105796813965 -``` -

-

- -  - -# HIAS UI - -Now that your classifier is setup and running, you can interact with it via the HIAS UI, and from other HIAS integrated applications. Before you can do so there is a final step to take on your server. - -![HIAS AI Inference](../assets/images/hias-ai-inference-endpoint.jpg) - -Head to the AI Agent page for your classifier on HIAS **(AI->Agents->List->Your Agent)**. On the edit page you will see the **Inference Endpoint**, you need to copy that value. - -## NGINX - -Now in console open the NGINX config file: - -``` -sudo nano /etc/nginx/sites-available/default -``` -Find **ADD NEW ENDPOINTS AFTER THIS NOTICE**, and add the following, replacing **YourEndpoint** with your inference endpoint value, and **YourIp/YourPort** with the IP/port of your Raspberry Pi. -``` -location ~ ^/AI/YourEndpoint/(.*)$ { - auth_basic "Restricted"; - auth_basic_user_file /etc/nginx/security/htpasswd; - proxy_pass http://YourIp:YourPort/$1; - } -``` -Save the file and exit, then run the following command: - -``` -sudo systemctl reload nginx -``` - -## Inference - -Now you are set up to communicate with the Acute Lymphoblastic Leukemia oneAPI Classifier from HIAS. Head to **(AI->Agents->List)** and then click on the **Inference** link. - -![HIAS AI Inference](../assets/images/hias-ai-inference.jpg) - -Once on the Inference page upload the twenty test images. Now make sure the server is running on the RPI and click the data to send it to the Acute Lymphoblastic Leukemia oneAPI Classifier for classification. - -### Verification - -As we know from the filenames in advance whether an image is negative or positive, we can compare the classification with the file name to check if a classification is a true/false positive, or a true/false negative. In the Diagnosis Results area Diagnosis represents the classification provided by the Acute Lymphoblastic Leukemia oneAPI Classifier, and Result provides the verification result. You should get the same results as when testing earlier back in the tutorial. The UI should 1 false negative and one false positive. - -  - -# Contributing - -The Peter Moss Acute Myeloid & Lymphoblastic Leukemia AI Research project encourages and youlcomes code contributions, bug fixes and enhancements from the Github. - -Please read the [CONTRIBUTING](../CONTRIBUTING.md "CONTRIBUTING") document for a full guide to forking our repositories and submitting your pull requests. You will also find information about our code of conduct on this page. - -## Contributors - -- [Adam Milton-Barker](https://www.leukemiaresearchassociation.ai/team/adam-milton-barker "Adam Milton-Barker") - [Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss](https://www.leukemiaresearchassociation.ai "Asociacion De Investigacion En Inteligencia Artificial Para La Leucemia Peter Moss") President/Founder & Lead Developer, Sabadell, Spain - -  - -# Versioning - -You use SemVer for versioning. For the versions available, see [Releases](../releases "Releases"). - -  - -# License - -This project is licensed under the **MIT License** - see the [LICENSE](../LICENSE "LICENSE") file for details. - -  - -# Bugs/Issues - -You use the [repo issues](../issues "repo issues") to track bugs and general requests related to using this project. See [CONTRIBUTING](../CONTRIBUTING.md "CONTRIBUTING") for more info on how to submit bugs, feature requests and proposals. \ No newline at end of file diff --git a/logs/.keep b/logs/.gitkeep similarity index 100% rename from logs/.keep rename to logs/.gitkeep diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..57b24aa --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,11 @@ +site_name: HIAS ALL oneAPI Classifier +site_url: https://www.leukemiaairesearch.com/research/project/peter-moss-medtech-research-project/hias-all-oneapi-classifier + +nav: + - Home: 'index.md' + - Installation: + - Installation (Ubuntu): 'installation/ubuntu.md' + - Installation (Raspbbery Pi): 'installation/raspberry-pi.md' + - Usage: + - Usage (Ubuntu): 'usage/ubuntu.md' + - Usage (Raspbbery Pi): 'usage/raspberry-pi.md' \ No newline at end of file diff --git a/model/freezing/frozen.pb b/model/freezing/frozen.pb index 6e49686..32e2de1 100644 Binary files a/model/freezing/frozen.pb and b/model/freezing/frozen.pb differ diff --git a/model/ir/.keep b/model/ir/.keep deleted file mode 100644 index e69de29..0000000 diff --git a/model/ir/frozen.bin b/model/ir/frozen.bin index d3b8428..e844cb9 100644 Binary files a/model/ir/frozen.bin and b/model/ir/frozen.bin differ diff --git a/model/ir/frozen.mapping b/model/ir/frozen.mapping index 4f58332..83e14ed 100644 --- a/model/ir/frozen.mapping +++ b/model/ir/frozen.mapping @@ -1,63 +1,59 @@ - - - - - - - - - - + + - - + + - - + + - - + + - - + + - - + + - + - - + + - - + + + + + + - - + + - - + + - - + + diff --git a/model/ir/frozen.xml b/model/ir/frozen.xml index 2004693..b557455 100644 --- a/model/ir/frozen.xml +++ b/model/ir/frozen.xml @@ -12,7 +12,7 @@ - + @@ -209,7 +209,7 @@ - + @@ -238,7 +238,7 @@ - + @@ -246,7 +246,7 @@ - + @@ -269,7 +269,7 @@ - + @@ -277,7 +277,7 @@ - + @@ -316,7 +316,7 @@ - + 1 diff --git a/model/plots/.keep b/model/plots/.keep deleted file mode 100644 index e69de29..0000000 diff --git a/model/plots/accuracy.png b/model/plots/accuracy.png index cbe6069..9ed0a5e 100644 Binary files a/model/plots/accuracy.png and b/model/plots/accuracy.png differ diff --git a/model/plots/auc.png b/model/plots/auc.png index 0dadf45..3f1b824 100644 Binary files a/model/plots/auc.png and b/model/plots/auc.png differ diff --git a/model/plots/confusion-matrix.png b/model/plots/confusion-matrix.png index 0a9916f..031c66a 100644 Binary files a/model/plots/confusion-matrix.png and b/model/plots/confusion-matrix.png differ diff --git a/model/plots/loss.png b/model/plots/loss.png index ebd2fc5..bb6a304 100644 Binary files a/model/plots/loss.png and b/model/plots/loss.png differ diff --git a/model/plots/precision.png b/model/plots/precision.png index 84f5a5d..832ed33 100644 Binary files a/model/plots/precision.png and b/model/plots/precision.png differ diff --git a/model/plots/recall.png b/model/plots/recall.png index 46b680b..371738e 100644 Binary files a/model/plots/recall.png and b/model/plots/recall.png differ diff --git a/model/saved_model.pb b/model/saved_model.pb index 3abd632..3059a93 100644 Binary files a/model/saved_model.pb and b/model/saved_model.pb differ diff --git a/model/variables/variables.data-00000-of-00001 b/model/variables/variables.data-00000-of-00001 index 66e6022..2f1e3ca 100644 Binary files a/model/variables/variables.data-00000-of-00001 and b/model/variables/variables.data-00000-of-00001 differ diff --git a/model/variables/variables.index b/model/variables/variables.index index 52aa42f..90949e1 100644 Binary files a/model/variables/variables.index and b/model/variables/variables.index differ diff --git a/model/weights.h5 b/model/weights.h5 index 338c6cd..2a180f4 100644 Binary files a/model/weights.h5 and b/model/weights.h5 differ diff --git a/modules/AbstractAgent.py b/modules/AbstractAgent.py index 1002525..31bbcd3 100644 --- a/modules/AbstractAgent.py +++ b/modules/AbstractAgent.py @@ -4,6 +4,11 @@ Represents a HIAS AI Agent. HIAS AI Agents process data using local AI models and communicate with HIAS IoT Agents using the MQTT protocol. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -23,7 +28,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-1 +- Adam Milton-Barker """ @@ -35,137 +40,130 @@ from modules.helpers import helpers from modules.mqtt import mqtt -from modules.model import model -#from modules.model_openvino import model_openvino from threading import Thread class AbstractAgent(ABC): - """ Abstract class representing a HIAS AI Agent. - - This object represents a HIAS AI Agent. HIAS AI Agents - process data using local AI models and communicate - with HIAS IoT Agents using the MQTT protocol. - - Attributes - ---------- - NA - - Methods - ------- - mqtt_conn() - Creates a MQTT connection with the HIAS iotJumpWay - private MQTT broker. - """ - - def __init__(self): - "Initializes the abstract_agent object." - super().__init__() - - self.mqtt = None - self.helpers = helpers("Agent") - self.confs = self.helpers.confs - self.credentials = self.helpers.credentials - self.model_type = None - - self.helpers.logger.info("Agent initialization complete.") - - def mqtt_conn(self, credentials): - """ Starts the HIAS iotJumpWay MQTT broker connection. """ - - self.mqtt = mqtt(self.helpers, "Agent", credentials) - self.mqtt.configure() - self.mqtt.start() - - self.mqtt.subscribe() - self.mqtt.commands_callback = self.mqtt_commands - - self.agent_threading() - - self.helpers.logger.info( - "HIAS iotJumpWay MQTT Broker connection created and subscriptions created.") - - def mqtt_start(self): - """ Starts the HIAS iotJumpWay MQTT broker connection. """ - - self.mqtt_conn({ - "host": self.credentials["iotJumpWay"]["host"], - "port": self.credentials["iotJumpWay"]["port"], - "location": self.credentials["iotJumpWay"]["location"], - "zone": self.credentials["iotJumpWay"]["zone"], - "entity": self.credentials["iotJumpWay"]["entity"], - "name": self.credentials["iotJumpWay"]["name"], - "un": self.credentials["iotJumpWay"]["un"], - "up": self.credentials["iotJumpWay"]["up"] - }) - - def mqtt_commands(self): - """ Called in the event of a command message from the HIAS iotJumpWay MQTT broker. """ - - def life(self): - """ Publishes entity statistics to HIAS. """ - - cpu = psutil.cpu_percent() - mem = psutil.virtual_memory()[2] - hdd = psutil.disk_usage('/').percent - - if self.model_type == "IR": - tmp = psutil.sensors_temperatures()['cpu_thermal'][0].current - else: - tmp = psutil.sensors_temperatures()['coretemp'][0].current - r = requests.get('http://ipinfo.io/json?token=' + - self.helpers.credentials["iotJumpWay"]["ipinfo"]) - data = r.json() - if data["status"] != 403: - location = data["loc"].split(',') - else: - location = [0, 0] - - self.mqtt.publish("Life", { - "CPU": float(cpu), - "Memory": float(mem), - "Diskspace": float(hdd), - "Temperature": float(tmp), - "Latitude": float(location[0]), - "Longitude": float(location[1]) - }) - - self.helpers.logger.info("Agent life statistics published.") - threading.Timer(300.0, self.life).start() - - def agent_threading(self): - """ Creates required module threads. """ - - # Life thread - threading.Timer(10.0, self.life).start() - - @abstractmethod - def set_model(self): - """ Creates & trains the model. """ - pass - - @abstractmethod - def train(self): - """ Creates & trains the model. """ - pass - - @abstractmethod - def load_model(self): - """ Loads model and classifies test data locally """ - pass - - @abstractmethod - def inference(self): - """ Loads model and classifies test data """ - pass - - @abstractmethod - def server(self): - """ Loads the API server """ - pass - - @abstractmethod - def inference_http(self): - """ Loads model and classifies test data via HTTP requests """ - pass + """ Abstract class representing a HIAS AI Agent. + + This object represents a HIAS AI Agent. HIAS AI Agents + process data using local AI models and communicate + with HIAS IoT Agents using the MQTT protocol. + """ + + def __init__(self): + "Initializes the abstract_agent object." + super().__init__() + + self.mqtt = None + self.helpers = helpers("Agent") + self.confs = self.helpers.confs + self.credentials = self.helpers.credentials + self.model_type = None + + self.helpers.logger.info( + "Agent initialization complete.") + + def mqtt_conn(self, credentials): + """ Starts the HIAS iotJumpWay MQTT broker connection. """ + + self.mqtt = mqtt( + self.helpers, "Agent", credentials) + self.mqtt.configure() + self.mqtt.start() + + self.mqtt.subscribe() + self.mqtt.commands_callback = self.mqtt_commands + + self.agent_threading() + + self.helpers.logger.info( + "HIAS iotJumpWay MQTT Broker connection created and subscriptions created.") + + def mqtt_start(self): + """ Starts the HIAS iotJumpWay MQTT broker connection. """ + + self.mqtt_conn({ + "host": self.credentials["iotJumpWay"]["host"], + "port": self.credentials["iotJumpWay"]["port"], + "location": self.credentials["iotJumpWay"]["location"], + "zone": self.credentials["iotJumpWay"]["zone"], + "entity": self.credentials["iotJumpWay"]["entity"], + "name": self.credentials["iotJumpWay"]["name"], + "un": self.credentials["iotJumpWay"]["un"], + "up": self.credentials["iotJumpWay"]["up"] + }) + + def mqtt_commands(self): + """ Called in the event of a command message from the HIAS iotJumpWay MQTT broker. """ + + def life(self): + """ Publishes entity statistics to HIAS. """ + + cpu = psutil.cpu_percent() + mem = psutil.virtual_memory()[2] + hdd = psutil.disk_usage('/').percent + + if self.model_type == "IR": + tmp = psutil.sensors_temperatures()['cpu_thermal'][0].current + else: + tmp = psutil.sensors_temperatures()['coretemp'][0].current + r = requests.get('http://ipinfo.io/json?token=' + + self.helpers.credentials["iotJumpWay"]["ipinfo"]) + data = r.json() + + if "loc" in data: + location = data["loc"].split(',') + else: + location = [0, 0] + + self.mqtt.publish("Life", { + "CPU": float(cpu), + "Memory": float(mem), + "Diskspace": float(hdd), + "Temperature": float(tmp), + "Latitude": float(location[0]), + "Longitude": float(location[1]) + }) + + self.helpers.logger.info( + "Agent life statistics published.") + threading.Timer(300.0, self.life).start() + + def agent_threading(self): + """ Creates required module threads. """ + + # Life thread + threading.Timer( + 10.0, self.life).start() + + @abstractmethod + def set_model(self): + """ Creates & trains the model. """ + pass + + @abstractmethod + def train(self): + """ Creates & trains the model. """ + pass + + @abstractmethod + def load_model(self): + """ Loads model and classifies test data locally """ + pass + + @abstractmethod + def inference(self): + """ Loads model and classifies test data """ + pass + + @abstractmethod + def server(self): + """ Loads the API server """ + pass + + @abstractmethod + def inference_http(self): + """ Loads model and classifies test data via HTTP requests """ + pass diff --git a/modules/AbstractData.py b/modules/AbstractData.py index 5b89639..bf438b8 100644 --- a/modules/AbstractData.py +++ b/modules/AbstractData.py @@ -4,6 +4,11 @@ Provides the HIAS AI Model with the required required data processing functionality. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -23,7 +28,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-1 +- Adam Milton-Barker """ @@ -36,70 +41,71 @@ class AbstractData(ABC): - """ HIAS AI Model Data Abstract Class. - - Provides the HIAS AI Model with the required required data - processing functionality. - """ - - def __init__(self, helpers): - "Initializes the AbstractData object." - super().__init__() - - self.helpers = helpers - self.confs = self.helpers.confs - - self.seed = self.helpers.confs["data"]["seed"] - self.dim = self.helpers.confs["data"]["dim"] - - seed(self.seed) - random.seed(self.seed) - - self.data = [] - self.labels = [] - - self.helpers.logger.info("Data class initialization complete.") - - def remove_testing(self): - """ Removes the testing images from the dataset. """ - - for img in self.helpers.confs["data"]["test_data"]: - original = "model/data/train/"+img - destination = "model/data/test/"+img - pathlib.Path(original).rename(destination) - self.helpers.logger.info(original + " moved to " + destination) - - @abstractmethod - def pre_process(self): - """ Processes the images. """ - pass - - @abstractmethod - def encode_labels(self): - """ One Hot Encodes the labels. """ - pass - - @abstractmethod - def convert_data(self): - """ Converts the training data to a numpy array. """ - pass - - @abstractmethod - def shuffle(self): - """ Shuffles the data and labels. """ - pass - - @abstractmethod - def get_split(self): - """ Splits the data and labels creating training and validation datasets. """ - pass - - @abstractmethod - def resize(self, path, dim): - """ Resizes an image to the provided dimensions (dim). """ - pass - - @abstractmethod - def reshape(self, img): - """ Reshapes an image. """ - pass + """ HIAS AI Model Data Abstract Class. + + Provides the HIAS AI Model with the required + required data processing functionality. + """ + + def __init__(self, helpers): + "Initializes the AbstractData object." + super().__init__() + + self.helpers = helpers + self.confs = self.helpers.confs + + self.seed = self.helpers.confs["data"]["seed"] + self.dim = self.helpers.confs["data"]["dim"] + + seed(self.seed) + random.seed(self.seed) + + self.data = [] + self.labels = [] + + self.helpers.logger.info("Data class initialization complete.") + + def remove_testing(self): + """ Removes the testing images from the dataset. """ + + for img in self.helpers.confs["data"]["test_data"]: + original = "model/data/train/"+img + destination = "model/data/test/"+img + pathlib.Path(original).rename(destination) + self.helpers.logger.info( + original + " moved to " + destination) + + @abstractmethod + def pre_process(self): + """ Processes the images. """ + pass + + @abstractmethod + def encode_labels(self): + """ One Hot Encodes the labels. """ + pass + + @abstractmethod + def convert_data(self): + """ Converts the training data to a numpy array. """ + pass + + @abstractmethod + def shuffle(self): + """ Shuffles the data and labels. """ + pass + + @abstractmethod + def get_split(self): + """ Splits the data and labels creating training and validation datasets. """ + pass + + @abstractmethod + def resize(self, path, dim): + """ Resizes an image to the provided dimensions (dim). """ + pass + + @abstractmethod + def reshape(self, img): + """ Reshapes an image. """ + pass diff --git a/modules/AbstractModel.py b/modules/AbstractModel.py index 21a347a..baa83d5 100644 --- a/modules/AbstractModel.py +++ b/modules/AbstractModel.py @@ -4,6 +4,11 @@ Represents a HIAS AI Model. HIAS AI Models are used by AI Agents to process incoming data. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -23,7 +28,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-1 +- Adam Milton-Barker """ @@ -43,137 +48,139 @@ from modules.data import data class AbstractModel(ABC): - """ Abstract class representing a HIAS AI Model. - - This object represents a HIAS AI Model. HIAS AI Models are used by AI Agents - to process incoming data. - """ - - def __init__(self, helpers): - "Initializes the AbstractModel object." - super().__init__() - - self.helpers = helpers - self.confs = self.helpers.confs - - os.environ["KMP_BLOCKTIME"] = "1" - os.environ["KMP_SETTINGS"] = "1" - os.environ["KMP_AFFINITY"] = "granularity=fine,verbose,compact,1,0" - os.environ["OMP_NUM_THREADS"] = str( - self.helpers.confs["agent"]["cores"]) - tf.config.threading.set_inter_op_parallelism_threads(1) - tf.config.threading.set_intra_op_parallelism_threads( - self.helpers.confs["agent"]["cores"]) - - self.data = data(self.helpers) - - self.testing_dir = self.helpers.confs["data"]["test"] - self.valid = self.helpers.confs["data"]["valid_types"] - self.seed = self.helpers.confs["data"]["seed"] - - random.seed(self.seed) - seed(self.seed) - tf.random.set_seed(self.seed) - - self.weights_file = self.helpers.confs["model"]["weights"] - self.model_json = self.helpers.confs["model"]["model"] - - self.helpers.logger.info("Model class initialization complete.") - - def http_request(self, img_path): - """ Sends image to the inference API endpoint. """ - - self.helpers.logger.info("Sending request for: " + img_path) - - _, img_encoded = cv2.imencode('.png', cv2.imread(img_path)) - response = requests.post(self.addr, data=img_encoded.tostring(), headers=self.headers) - response = json.loads(response.text) - - return response - - @abstractmethod - def prepare_data(self): - """ Prepares the model data """ - pass - - @abstractmethod - def prepare_network(self): - """ Builds the network """ - pass - - @abstractmethod - def train(self): - """ Trains the model """ - pass - - @abstractmethod - def freeze_model(self): - """ Freezes the model """ - pass - - @abstractmethod - def save_model_as_json(self): - """ Saves the model as JSON """ - pass - - @abstractmethod - def save_weights(self): - """ Saves the model weights """ - pass - - @abstractmethod - def load(self): - """ Loads the model """ - pass - - @abstractmethod - def visualize_metrics(self): - """ Visualize the metrics. """ - pass - - @abstractmethod - def confusion_matrix(self): - """ Prints/displays the confusion matrix. """ - pass - - @abstractmethod - def figures_of_merit(self): - """ Calculates/prints the figures of merit. """ - pass - - @abstractmethod - def evaluate(self): - """ Evaluates the model """ - pass - - @abstractmethod - def predictions(self): - """ Makes predictions on the train & test sets. """ - pass - - @abstractmethod - def predict(self, img): - """ Gets a prediction for an image. """ - pass - - @abstractmethod - def reshape(self, img): - """ Reshapes the image. """ - pass - - @abstractmethod - def test(self): - """Local test mode - - Loops through the test directory and classifies the images. - """ - pass - - @abstractmethod - def test_http(self): - """ HTTP test mode - - Loops through the test directory and classifies the images - by sending data to the classifier using HTTP requests. - """ - pass + """ Abstract class representing a HIAS AI Model. + + This object represents a HIAS AI Model. HIAS AI + Models are used by AI Agents to process incoming data. + """ + + def __init__(self, helpers): + "Initializes the AbstractModel object." + super().__init__() + + self.helpers = helpers + self.confs = self.helpers.confs + + os.environ["KMP_BLOCKTIME"] = "1" + os.environ["KMP_SETTINGS"] = "1" + os.environ["KMP_AFFINITY"] = "granularity=fine,verbose,compact,1,0" + os.environ["OMP_NUM_THREADS"] = str( + self.helpers.confs["agent"]["cores"]) + tf.config.threading.set_inter_op_parallelism_threads(1) + tf.config.threading.set_intra_op_parallelism_threads( + self.helpers.confs["agent"]["cores"]) + + self.data = data(self.helpers) + + self.testing_dir = self.helpers.confs["data"]["test"] + self.valid = self.helpers.confs["data"]["valid_types"] + self.seed = self.helpers.confs["data"]["seed"] + + random.seed(self.seed) + seed(self.seed) + tf.random.set_seed(self.seed) + + self.weights_file = self.helpers.confs["model"]["weights"] + self.model_json = self.helpers.confs["model"]["model"] + + self.helpers.logger.info( + "Model class initialization complete.") + + def http_request(self, img_path): + """ Sends image to the inference API endpoint. """ + + self.helpers.logger.info("Sending request for: " + img_path) + + _, img_encoded = cv2.imencode('.jpg', cv2.imread(img_path)) + response = requests.post( + self.addr, data=img_encoded.tostring(), headers=self.headers) + response = json.loads(response.text) + + return response + + @abstractmethod + def prepare_data(self): + """ Prepares the model data """ + pass + + @abstractmethod + def prepare_network(self): + """ Builds the network """ + pass + + @abstractmethod + def train(self): + """ Trains the model """ + pass + + @abstractmethod + def freeze_model(self): + """ Freezes the model """ + pass + + @abstractmethod + def save_model_as_json(self): + """ Saves the model as JSON """ + pass + + @abstractmethod + def save_weights(self): + """ Saves the model weights """ + pass + + @abstractmethod + def load(self): + """ Loads the model """ + pass + + @abstractmethod + def visualize_metrics(self): + """ Visualize the metrics. """ + pass + + @abstractmethod + def confusion_matrix(self): + """ Prints/displays the confusion matrix. """ + pass + + @abstractmethod + def figures_of_merit(self): + """ Calculates/prints the figures of merit. """ + pass + + @abstractmethod + def evaluate(self): + """ Evaluates the model """ + pass + + @abstractmethod + def predictions(self): + """ Makes predictions on the train & test sets. """ + pass + + @abstractmethod + def predict(self, img): + """ Gets a prediction for an image. """ + pass + + @abstractmethod + def reshape(self, img): + """ Reshapes the image. """ + pass + + @abstractmethod + def test(self): + """Local test mode + + Loops through the test directory and classifies the images. + """ + pass + + @abstractmethod + def test_http(self): + """ HTTP test mode + + Loops through the test directory and classifies the images + by sending data to the classifier using HTTP requests. + """ + pass diff --git a/modules/AbstractOpenVINO.py b/modules/AbstractOpenVINO.py index 05534f5..a62e3c7 100644 --- a/modules/AbstractOpenVINO.py +++ b/modules/AbstractOpenVINO.py @@ -1,8 +1,13 @@ #!/usr/bin/env python """ Abstract class representing a HIAS AI OpenVINO Model. -Represents a HIAS AI OpenVINO Model. HIAS AI OpenVINO Models are used by AI Agents to process -incoming data. +Represents a HIAS AI OpenVINO Model. HIAS AI OpenVINO +Models are used by AI Agents to process incoming data. + +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal @@ -23,7 +28,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-3 +- Adam Milton-Barker """ @@ -36,76 +41,78 @@ class AbstractOpenVINO(ABC): - """ Abstract class representing a HIAS AI OpenVINO Model. + """ Abstract class representing a HIAS AI OpenVINO Model. - This object represents a HIAS AI Model. HIAS AI Models are used by AI Agents - to process incoming data. - """ + This object represents a HIAS AI Model. HIAS AI Models + are used by AI Agents to process incoming data. + """ - def __init__(self, helpers): - "Initializes the AbstractOpenVINO object." - super().__init__() + def __init__(self, helpers): + "Initializes the AbstractOpenVINO object." + super().__init__() - self.helpers = helpers - self.confs = self.helpers.confs + self.helpers = helpers + self.confs = self.helpers.confs - self.testing_dir = self.helpers.confs["data"]["test"] - self.valid = self.helpers.confs["data"]["valid_types"] + self.testing_dir = self.helpers.confs["data"]["test"] + self.valid = self.helpers.confs["data"]["valid_types"] - self.imsize = self.helpers.confs["data"]["dim"] + self.imsize = self.helpers.confs["data"]["dim"] - self.helpers.logger.info("OpenVINO class initialization complete.") + self.helpers.logger.info( + "OpenVINO class initialization complete.") - def http_request(self, img_path): - """ Sends image to the inference API endpoint. """ + def http_request(self, img_path): + """ Sends image to the inference API endpoint. """ - self.helpers.logger.info("Sending request for: " + img_path) + self.helpers.logger.info( + "Sending request for: " + img_path) - _, img_encoded = cv2.imencode('.png', cv2.imread(img_path)) - response = requests.post( - self.addr, data=img_encoded.tostring(), headers=self.headers) - response = json.loads(response.text) + _, img_encoded = cv2.imencode('.png', cv2.imread(img_path)) + response = requests.post( + self.addr, data=img_encoded.tostring(), headers=self.headers) + response = json.loads(response.text) - return response + return response - @abstractmethod - def load(self): - """ Loads the model """ - pass + @abstractmethod + def load(self): + """ Loads the model """ + pass - @abstractmethod - def setBlob(self, frame): - """ Gets a blob from the color frame """ - pass + @abstractmethod + def setBlob(self, frame): + """ Gets a blob from the color frame """ + pass - @abstractmethod - def forwardPass(self): - """ Gets a blob from the color frame """ - pass + @abstractmethod + def forwardPass(self): + """ Gets a blob from the color frame """ + pass - @abstractmethod - def predict(self): - """ Gets a prediction for an image. """ - pass + @abstractmethod + def predict(self): + """ Gets a prediction for an image. """ + pass - @abstractmethod - def test(self): - """Local test mode + @abstractmethod + def test(self): + """Local test mode - Loops through the test directory and classifies the images. - """ - pass + Loops through the test directory and classifies the images. + """ + pass - @abstractmethod - def test_http(self): - """ HTTP test mode + @abstractmethod + def test_http(self): + """ HTTP test mode - Loops through the test directory and classifies the images - by sending data to the classifier using HTTP requests. - """ - pass + Loops through the test directory and classifies the images + by sending data to the classifier using HTTP requests. + """ + pass - @abstractmethod - def resize(self, img): - """ Resizes an image. """ - pass + @abstractmethod + def resize(self, img): + """ Resizes an image. """ + pass diff --git a/modules/AbstractServer.py b/modules/AbstractServer.py index 79d4f70..e03185c 100644 --- a/modules/AbstractServer.py +++ b/modules/AbstractServer.py @@ -3,6 +3,11 @@ Abstract class for the HIAS IoT Agent server/API. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -22,45 +27,46 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-2 +- Adam Milton-Barker """ from abc import ABC, abstractmethod class AbstractServer(ABC): - """ Server/API abstract class. + """ Server/API abstract class. - Abstract class for the HIAS IoT Agent server/API. - """ + Abstract class for the HIAS IoT Agent server/API. + """ - def __init__(self, helpers, model, model_type, mqtt): - "Initializes the AbstractModel object." - super().__init__() + def __init__(self, helpers, model, model_type, mqtt): + "Initializes the AbstractModel object." + super().__init__() - self.helpers = helpers - self.confs = self.helpers.confs + self.helpers = helpers + self.confs = self.helpers.confs - self.model = model - self.model_type = model_type + self.model = model + self.model_type = model_type - if mqtt is not None: - self.mqtt = mqtt + if mqtt is not None: + self.mqtt = mqtt - self.helpers.logger.info("Agent initialization complete.") + self.helpers.logger.info( + "Agent initialization complete.") - @abstractmethod - def predict(self, req): - """ Classifies an image sent via HTTP. """ - pass + @abstractmethod + def predict(self, req): + """ Classifies an image sent via HTTP. """ + pass - @abstractmethod - def predict_openvino(self, req): - """ Classifies an image sent via HTTP using OpenVINO. """ - pass + @abstractmethod + def predict_openvino(self, req): + """ Classifies an image sent via HTTP using OpenVINO. """ + pass - @abstractmethod - def start(self, img_path): - """ Sends image to the inference API endpoint. """ - pass + @abstractmethod + def start(self, img_path): + """ Sends image to the inference API endpoint. """ + pass diff --git a/modules/augmentation.py b/modules/augmentation.py index f16bf50..f056587 100644 --- a/modules/augmentation.py +++ b/modules/augmentation.py @@ -3,6 +3,11 @@ Provides data augmentation methods. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -36,89 +41,82 @@ from skimage import transform as tm class augmentation(): - """ HIAS AI Model Data Augmentation Class - - Provides data augmentation methods. - """ - - def __init__(self, helpers): - """ Initializes the class. """ + """ HIAS AI Model Data Augmentation Class - self.helpers = helpers + Provides data augmentation methods. + """ - self.seed = self.helpers.confs["data"]["seed"] - seed(self.seed) + def __init__(self, helpers): + """ Initializes the class. """ - self.helpers.logger.info("Augmentation class initialization complete.") + self.helpers = helpers - def grayscale(self, data): - """ Creates a grayscale copy. """ + self.seed = self.helpers.confs["data"]["seed"] + seed(self.seed) - gray = cv2.cvtColor(data, cv2.COLOR_BGR2GRAY) - return np.dstack([gray, gray, gray]).astype(np.float32)/255. + self.helpers.logger.info( + "Augmentation class initialization complete.") - def equalize_hist(self, data): - """ Creates a histogram equalized copy. + def grayscale(self, data): + """ Creates a grayscale copy. """ - Credit: Amita Kapoor & Taru Jain - Exploring novel convolutional network architecture to build a classification - system for better assistance in diagonosing Acute Lymphoblastic Leukemia in - blood cells. - https://github.com/AMLResearchProject/ALL-Keras-2019 - """ + gray = cv2.cvtColor(data, cv2.COLOR_BGR2GRAY) + return np.dstack([gray, gray, gray]).astype(np.float32)/255. - img_to_yuv = cv2.cvtColor(data, cv2.COLOR_BGR2YUV) - img_to_yuv[:, :, 0] = cv2.equalizeHist(img_to_yuv[:, :, 0]) - hist_equalization_result = cv2.cvtColor(img_to_yuv, cv2.COLOR_YUV2BGR) - return hist_equalization_result.astype(np.float32)/255. + def equalize_hist(self, data): + """ Creates a histogram equalized copy. """ - def reflection(self, data): - """ Creates a reflected copy. """ + img_to_yuv = cv2.cvtColor(data, cv2.COLOR_BGR2YUV) + img_to_yuv[:, :, 0] = cv2.equalizeHist(img_to_yuv[:, :, 0]) + hist_equalization_result = cv2.cvtColor(img_to_yuv, cv2.COLOR_YUV2BGR) + return hist_equalization_result.astype(np.float32)/255. - return cv2.flip(data, 0).astype(np.float32)/255., cv2.flip(data, 1).astype(np.float32)/255. + def reflection(self, data): + """ Creates a reflected copy. """ - def gaussian(self, data): - """ Creates a gaussian blurred copy. """ + return cv2.flip(data, 0).astype(np.float32)/255., cv2.flip(data, 1).astype(np.float32)/255. - return ndimage.gaussian_filter(data, sigma=5.11).astype(np.float32)/255. + def gaussian(self, data): + """ Creates a gaussian blurred copy. """ - def translate(self, data): - """ Creates transformed copy. """ + return ndimage.gaussian_filter( + data, sigma=5.11).astype(np.float32)/255. - cols, rows, chs = data.shape + def translate(self, data): + """ Creates transformed copy. """ - return cv2.warpAffine(data, np.float32([[1, 0, 84], [0, 1, 56]]), (rows, cols), - borderMode=cv2.BORDER_CONSTANT, borderValue=(144, 159, 162)).astype(np.float32)/255. + cols, rows, chs = data.shape - def rotation(self, data, label, tdata, tlabels): - """ Creates rotated copies. """ + return cv2.warpAffine( + data, np.float32([[1, 0, 84], [0, 1, 56]]), (rows, cols), + borderMode=cv2.BORDER_CONSTANT, + borderValue=(144, 159, 162)).astype(np.float32)/255. - cols, rows, chs = data.shape + def rotation(self, data, label, tdata, tlabels): + """ Creates rotated copies. """ - for i in range(0, self.helpers.confs["data"]["rotations"]): - # Seed needs to be set each time randint is called - random.seed(self.seed) - rand_deg = random.randint(-180, 180) - matrix = cv2.getRotationMatrix2D((cols/2, rows/2), rand_deg, 0.70) - rotated = cv2.warpAffine(data, matrix, (rows, cols), borderMode=cv2.BORDER_CONSTANT, - borderValue=(144, 159, 162)) + cols, rows, chs = data.shape - rotated = rotated.astype(np.float32)/255. + for i in range(0, self.helpers.confs["data"]["rotations"]): + # Seed needs to be set each time randint is called + random.seed(self.seed) + rand_deg = random.randint(-180, 180) + matrix = cv2.getRotationMatrix2D( + (cols/2, rows/2), rand_deg, 0.70) + rotated = cv2.warpAffine( + data, matrix, (rows, cols), + borderMode=cv2.BORDER_CONSTANT, + borderValue=(144, 159, 162)) - tdata.append(rotated) - tlabels.append(label) + rotated = rotated.astype(np.float32)/255. - return tdata, tlabels + tdata.append(rotated) + tlabels.append(label) - def shear(self, data): - """ Creates a histogram equalized copy. + return tdata, tlabels - Credit: Amita Kapoor & Taru Jain - Exploring novel convolutional network architecture to build a classification - system for better assistance in diagonosing Acute Lymphoblastic Leukemia in - blood cells. - https://github.com/AMLResearchProject/ALL-Keras-2019 - """ + def shear(self, data): + """ Creates a histogram equalized copy. """ - at = tm.AffineTransform(shear=0.5) - return tm.warp(data, inverse_map=at) + at = tm.AffineTransform(shear=0.5) + return tm.warp(data, inverse_map=at) diff --git a/modules/data.py b/modules/data.py index b94f38e..9e2b42a 100644 --- a/modules/data.py +++ b/modules/data.py @@ -4,6 +4,11 @@ Provides the HIAS AI Model with the required required data processing functionality. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -43,131 +48,138 @@ from modules.augmentation import augmentation class data(AbstractData): - """ HIAS AI Model Data Class. + """ HIAS AI Model Data Class. - Provides the HIAS AI Model with the required required data - processing functionality. - """ + Provides the HIAS AI Model with the required + required data processing functionality. + """ - def pre_process(self): - """ Processes the images. """ + def pre_process(self): + """ Processes the images. """ - aug = augmentation(self.helpers) + aug = augmentation(self.helpers) - data_dir = pathlib.Path( - self.helpers.confs["data"]["train_dir"]) - data = list(data_dir.glob( - '*' + self.helpers.confs["data"]["file_type"])) + data_dir = pathlib.Path( + self.helpers.confs["data"]["train_dir"]) + data = list(data_dir.glob( + '*' + self.helpers.confs["data"]["file_type"])) - count = 0 - neg_count = 0 - pos_count = 0 + count = 0 + neg_count = 0 + pos_count = 0 - augmented_data = [] - augmented_labels = [] + augmented_data = [] + augmented_labels = [] - for rimage in data: - fpath = str(rimage) - fname = os.path.basename(rimage) - label = 0 if "_0" in fname else 1 + for rimage in data: + fpath = str(rimage) + fname = os.path.basename(rimage) + label = 0 if "_0" in fname else 1 - image = self.resize(fpath, self.dim) + image = self.resize(fpath, self.dim) - if image.shape[2] == 1: - image = np.dstack( - [image, image, image]) + if image.shape[2] == 1: + image = np.dstack( + [image, image, image]) - augmented_data.append(image.astype(np.float32)/255.) - augmented_labels.append(label) + augmented_data.append(image.astype(np.float32)/255.) + augmented_labels.append(label) - augmented_data.append(aug.grayscale(image)) - augmented_labels.append(label) + augmented_data.append(aug.grayscale(image)) + augmented_labels.append(label) - augmented_data.append(aug.equalize_hist(image)) - augmented_labels.append(label) + augmented_data.append(aug.equalize_hist(image)) + augmented_labels.append(label) - horizontal, vertical = aug.reflection(image) - augmented_data.append(horizontal) - augmented_labels.append(label) + horizontal, vertical = aug.reflection(image) + augmented_data.append(horizontal) + augmented_labels.append(label) - augmented_data.append(vertical) - augmented_labels.append(label) + augmented_data.append(vertical) + augmented_labels.append(label) - augmented_data.append(aug.gaussian(image)) - augmented_labels.append(label) + augmented_data.append(aug.gaussian(image)) + augmented_labels.append(label) - augmented_data.append(aug.translate(image)) - augmented_labels.append(label) + augmented_data.append(aug.translate(image)) + augmented_labels.append(label) - augmented_data.append(aug.shear(image)) - augmented_labels.append(label) + augmented_data.append(aug.shear(image)) + augmented_labels.append(label) - self.data, self.labels = aug.rotation( - image, label, augmented_data, augmented_labels) + self.data, self.labels = aug.rotation( + image, label, augmented_data, augmented_labels) - if "_0" in fname: - neg_count += 9 - else: - pos_count += 9 - count += 9 + if "_0" in fname: + neg_count += 9 + else: + pos_count += 9 + count += 9 - self.shuffle() - self.convert_data() - self.encode_labels() + self.shuffle() + self.convert_data() + self.encode_labels() - self.helpers.logger.info("Raw data: " + str(count)) - self.helpers.logger.info("Raw negative data: " + str(neg_count)) - self.helpers.logger.info("Raw positive data: " + str(pos_count)) - self.helpers.logger.info("Augmented data: " + str(self.data.shape)) - self.helpers.logger.info("Labels: " + str(self.labels.shape)) + self.helpers.logger.info("Raw data: " + str(count)) + self.helpers.logger.info("Raw negative data: " + str(neg_count)) + self.helpers.logger.info("Raw positive data: " + str(pos_count)) + self.helpers.logger.info("Augmented data: " + str(self.data.shape)) + self.helpers.logger.info("Labels: " + str(self.labels.shape)) - self.get_split() + self.get_split() - def convert_data(self): - """ Converts the training data to a numpy array. """ + def convert_data(self): + """ Converts the training data to a numpy array. """ - self.data = np.array(self.data) - self.helpers.logger.info("Data shape: " + str(self.data.shape)) + self.data = np.array(self.data) + self.helpers.logger.info( + "Data shape: " + str(self.data.shape)) - def encode_labels(self): - """ One Hot Encodes the labels. """ + def encode_labels(self): + """ One Hot Encodes the labels. """ - encoder = OneHotEncoder(categories='auto') + encoder = OneHotEncoder(categories='auto') - self.labels = np.reshape(self.labels, (-1, 1)) - self.labels = encoder.fit_transform(self.labels).toarray() - self.helpers.logger.info("Labels shape: " + str(self.labels.shape)) + self.labels = np.reshape(self.labels, (-1, 1)) + self.labels = encoder.fit_transform(self.labels).toarray() + self.helpers.logger.info( + "Labels shape: " + str(self.labels.shape)) - def shuffle(self): - """ Shuffles the data and labels. """ + def shuffle(self): + """ Shuffles the data and labels. """ - self.data, self.labels = shuffle( - self.data, self.labels, random_state=self.seed) + self.data, self.labels = shuffle( + self.data, self.labels, random_state=self.seed) - def get_split(self): - """ Splits the data and labels creating training and validation datasets. """ + def get_split(self): + """ Splits the data and labels creating training and validation datasets. """ - self.X_train, self.X_test, self.y_train, self.y_test = train_test_split( - self.data, self.labels, test_size=0.255, random_state=self.seed) + self.X_train, self.X_test, self.y_train, self.y_test = train_test_split( + self.data, self.labels, test_size=0.255, random_state=self.seed) - self.helpers.logger.info("Training data: " + str(self.X_train.shape)) - self.helpers.logger.info("Training labels: " + str(self.y_train.shape)) - self.helpers.logger.info("Validation data: " + str(self.X_test.shape)) - self.helpers.logger.info("Validation labels: " + str(self.y_test.shape)) + self.helpers.logger.info( + "Training data: " + str(self.X_train.shape)) + self.helpers.logger.info( + "Training labels: " + str(self.y_train.shape)) + self.helpers.logger.info( + "Validation data: " + str(self.X_test.shape)) + self.helpers.logger.info( + "Validation labels: " + str(self.y_test.shape)) - def resize(self, path, dim): - """ Resizes an image to the provided dimensions (dim). """ + def resize(self, path, dim): + """ Resizes an image to the provided dimensions (dim). """ - return cv2.resize(cv2.imread(path), (dim, dim)) + return cv2.resize( + cv2.imread(path), (dim, dim)) - def reshape(self, img): - """ Classifies an image sent via HTTP. """ + def reshape(self, img): + """ Classifies an image sent via HTTP. """ - n, c, h, w = [1, 3, self.confs["data"]["dim"], - self.confs["data"]["dim"]] - processed = img.resize((h, w), resample=Image.BILINEAR) - processed = (np.array(processed) - 0) / 255.0 - processed = processed.transpose((2, 0, 1)) - processed = processed.reshape((n, h, w, c)) + n, c, h, w = [1, 3, self.confs["data"]["dim"], + self.confs["data"]["dim"]] + processed = img.resize((h, w), resample=Image.BILINEAR) + processed = (np.array(processed) - 0) / 255.0 + processed = processed.transpose((2, 0, 1)) + processed = processed.reshape((n, h, w, c)) - return processed + return processed diff --git a/modules/helpers.py b/modules/helpers.py index ca316c1..c0743b1 100644 --- a/modules/helpers.py +++ b/modules/helpers.py @@ -2,6 +2,11 @@ Configuration and logging functions. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -21,7 +26,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-1 +- Adam Milton-Barker """ @@ -30,62 +35,81 @@ import json import os import sys -import time +import socket from datetime import datetime class helpers(): - """ Helper Class - - HIAS AI Agent helper functions. - """ - - def __init__(self, ltype, log=True): - """ Initializes the Helpers Class. """ - - # Loads system configs - self.confs = {} - self.loadConfs() - - # Sets system logging - self.logger = logging.getLogger(ltype) - self.logger.setLevel(logging.INFO) - - formatter = logging.Formatter( - '%(asctime)s - %(name)s - %(levelname)s - %(message)s') - - allLogHandler = handlers.TimedRotatingFileHandler( - os.path.dirname(os.path.abspath(__file__)) + '/../logs/all.log', when='H', interval=1, backupCount=0) - allLogHandler.setLevel(logging.INFO) - allLogHandler.setFormatter(formatter) - - errorLogHandler = handlers.TimedRotatingFileHandler( - os.path.dirname(os.path.abspath(__file__)) + '/../logs/error.log', when='H', interval=1, backupCount=0) - errorLogHandler.setLevel(logging.ERROR) - errorLogHandler.setFormatter(formatter) - - warningLogHandler = handlers.TimedRotatingFileHandler( - os.path.dirname(os.path.abspath(__file__)) + '/../logs/warning.log', when='H', interval=1, backupCount=0) - warningLogHandler.setLevel(logging.WARNING) - warningLogHandler.setFormatter(formatter) - - consoleHandler = logging.StreamHandler(sys.stdout) - consoleHandler.setFormatter(formatter) - - self.logger.addHandler(allLogHandler) - self.logger.addHandler(errorLogHandler) - self.logger.addHandler(warningLogHandler) - self.logger.addHandler(consoleHandler) - - if log is True: - self.logger.info("Helpers class initialization complete.") - - def loadConfs(self): - """ Load the configuration. """ - - with open(os.path.dirname(os.path.abspath(__file__)) + '/../configuration/credentials.json') as credentials: - self.credentials = json.loads(credentials.read()) - - with open(os.path.dirname(os.path.abspath(__file__)) + '/../configuration/config.json') as confs: - self.confs = json.loads(confs.read()) + """ Helper Class + + HIAS AI Agent helper functions. + """ + + def __init__(self, ltype, log=True): + """ Initializes the Helpers Class. """ + + # Loads system configs + self.confs = {} + self.load_confs() + + # Sets system logging + self.logger = logging.getLogger(ltype) + self.logger.setLevel(logging.INFO) + + formatter = logging.Formatter( + '%(asctime)s - %(name)s - %(levelname)s - %(message)s') + + allLogHandler = handlers.TimedRotatingFileHandler( + os.path.dirname( + os.path.abspath(__file__)) + '/../logs/all.log', + when='H', interval=1, backupCount=0) + allLogHandler.setLevel(logging.INFO) + allLogHandler.setFormatter(formatter) + + errorLogHandler = handlers.TimedRotatingFileHandler( + os.path.dirname( + os.path.abspath(__file__)) + '/../logs/error.log', + when='H', interval=1, backupCount=0) + errorLogHandler.setLevel(logging.ERROR) + errorLogHandler.setFormatter(formatter) + + warningLogHandler = handlers.TimedRotatingFileHandler( + os.path.dirname( + os.path.abspath(__file__)) + '/../logs/warning.log', + when='H', interval=1, backupCount=0) + warningLogHandler.setLevel(logging.WARNING) + warningLogHandler.setFormatter(formatter) + + consoleHandler = logging.StreamHandler(sys.stdout) + consoleHandler.setFormatter(formatter) + + self.logger.addHandler(allLogHandler) + self.logger.addHandler(errorLogHandler) + self.logger.addHandler(warningLogHandler) + self.logger.addHandler(consoleHandler) + + if log is True: + self.logger.info( + "Helpers class initialization complete.") + + def load_confs(self): + """ Load the configuration. """ + + with open(os.path.dirname( + os.path.abspath(__file__)) + '/../configuration/credentials.json') as credentials: + self.credentials = json.loads(credentials.read()) + + with open( + os.path.dirname(os.path.abspath(__file__)) + '/../configuration/config.json') as confs: + self.confs = json.loads(confs.read()) + + def get_ip_addr(self): + """ Load the configuration. """ + + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + s.connect(("8.8.8.8", 80)) + ipaddr = s.getsockname()[0] + s.close() + + return ipaddr diff --git a/modules/model.py b/modules/model.py index 23c86d2..77767d5 100644 --- a/modules/model.py +++ b/modules/model.py @@ -4,6 +4,11 @@ Represents a HIAS AI Model. HIAS AI Models are used by AI Agents to process incoming data. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -23,7 +28,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-1 +- Adam Milton-Barker """ @@ -45,391 +50,422 @@ class model(AbstractModel): - """ Class representing a HIAS AI Model. - - This object represents a HIAS AI Model.HIAS AI Models are used by AI Agents - to process incoming data. - """ - - def prepare_data(self): - """ Creates/sorts dataset. """ - - self.data.remove_testing() - self.data.pre_process() - - self.helpers.logger.info("Data preperation complete.") - - def prepare_network(self): - """ Builds the network. - - Replicates the networked outlined in the Acute Leukemia Classification - Using Convolution Neural Network In Clinical Decision Support System paper - using Tensorflow 2.0. - https://airccj.org/CSCP/vol7/csit77505.pdf - """ - - self.val_steps = self.helpers.confs["train"]["val_steps"] - self.batch_size = self.helpers.confs["train"]["batch"] - self.epochs = self.helpers.confs["train"]["epochs"] - - self.tf_model = tf.keras.models.Sequential([ - tf.keras.layers.ZeroPadding2D( - padding=(2, 2), input_shape=self.data.X_train.shape[1:]), - tf.keras.layers.Conv2D(30, (5, 5), strides=1, - padding="valid", activation='relu'), - tf.keras.layers.ZeroPadding2D(padding=(2, 2)), - tf.keras.layers.Conv2D(30, (5, 5), strides=1, - padding="valid", activation='relu'), - tf.keras.layers.MaxPooling2D( - pool_size=(2, 2), strides=2, padding='valid'), - tf.keras.layers.Flatten(), - tf.keras.layers.Dense(2), - tf.keras.layers.Activation('softmax') - ], - "AllOneApiClassifier") - self.tf_model.summary() - - self.helpers.logger.info("Network initialization complete.") - - def train(self): - """ Trains the model - - Trains the neural network. - """ - - self.helpers.logger.info("Using Adam Optimizer.") - optimizer = tf.keras.optimizers.Adam(lr=self.helpers.confs["train"]["learning_rate_adam"], - decay = self.helpers.confs["train"]["decay_adam"]) - - self.tf_model.compile(optimizer=optimizer, - loss='binary_crossentropy', - metrics=[tf.keras.metrics.BinaryAccuracy(name='acc'), - tf.keras.metrics.Precision(name='precision'), - tf.keras.metrics.Recall(name='recall'), - tf.keras.metrics.AUC(name='auc') ]) - - self.history = self.tf_model.fit(self.data.X_train, self.data.y_train, - validation_data=(self.data.X_test, self.data.y_test), - validation_steps=self.val_steps, - epochs=self.epochs) - - print(self.history) - print("") - - self.freeze_model() - self.save_model_as_json() - self.save_weights() - - def freeze_model(self): - """ Freezes the model """ - - tf.saved_model.save( - self.tf_model, self.helpers.confs["model"]["saved_model_dir"]) - - fmodel = tf.function(lambda x: self.tf_model(x)) - fmodel = fmodel.get_concrete_function( - x=tf.TensorSpec(self.tf_model.inputs[0].shape, self.tf_model.inputs[0].dtype)) - - freeze = convert_variables_to_constants_v2(fmodel) - freeze.graph.as_graph_def() - - layers = [op.name for op in freeze.graph.get_operations()] - self.helpers.logger.info("Frozen model layers") - for layer in layers: - self.helpers.logger.info(layer) - - self.helpers.logger.info("Frozen model inputs") - self.helpers.logger.info(freeze.inputs) - self.helpers.logger.info("Frozen model outputs") - self.helpers.logger.info(freeze.outputs) - - tf.io.write_graph(graph_or_graph_def=freeze.graph, - logdir=self.helpers.confs["model"]["freezing_log_dir"], - name=self.helpers.confs["model"]["frozen"], - as_text=False) - - def save_model_as_json(self): - """ Saves the model as JSON """ - - with open(self.model_json, "w") as file: - file.write(self.tf_model.to_json()) - - self.helpers.logger.info("Model JSON saved " + self.model_json) - - def save_weights(self): - """ Saves the model weights """ - - self.tf_model.save_weights(self.weights_file) - self.helpers.logger.info("Weights saved " + self.weights_file) - - def visualize_metrics(self): - """ Visualize the metrics. """ - - plt.plot(self.history.history['acc']) - plt.plot(self.history.history['val_acc']) - plt.title('Model Accuracy') - plt.ylabel('Accuracy') - plt.xlabel('Epoch') - plt.ylim((0, 1)) - plt.legend(['Train', 'Validate'], loc='upper left') - plt.savefig('model/plots/accuracy.png') - plt.show() - plt.clf() - - plt.plot(self.history.history['loss']) - plt.plot(self.history.history['val_loss']) - plt.title('Model Loss') - plt.ylabel('loss') - plt.xlabel('Epoch') - plt.legend(['Train', 'Validate'], loc='upper left') - plt.savefig('model/plots/loss.png') - plt.show() - plt.clf() - - plt.plot(self.history.history['auc']) - plt.plot(self.history.history['val_auc']) - plt.title('Model AUC') - plt.ylabel('AUC') - plt.xlabel('Epoch') - plt.legend(['Train', 'Validate'], loc='upper left') - plt.savefig('model/plots/auc.png') - plt.show() - plt.clf() - - plt.plot(self.history.history['precision']) - plt.plot(self.history.history['val_precision']) - plt.title('Model Precision') - plt.ylabel('Precision') - plt.xlabel('Epoch') - plt.legend(['Train', 'Validate'], loc='upper left') - plt.savefig('model/plots/precision.png') - plt.show() - plt.clf() - - plt.plot(self.history.history['recall']) - plt.plot(self.history.history['val_recall']) - plt.title('Model Recall') - plt.ylabel('Recall') - plt.xlabel('Epoch') - plt.legend(['Train', 'Validate'], loc='upper left') - plt.savefig('model/plots/recall.png') - plt.show() - plt.clf() - - def confusion_matrix(self): - """ Prints/displays the confusion matrix. """ - - self.matrix = confusion_matrix(self.data.y_test.argmax(axis=1), - self.test_preds.argmax(axis=1)) - - self.helpers.logger.info("Confusion Matrix: " + str(self.matrix)) - print("") - - plot_confusion_matrix(conf_mat=self.matrix) - plt.savefig('model/plots/confusion-matrix.png') - plt.show() - plt.clf() - - def figures_of_merit(self): - """ Calculates/prints the figures of merit. - - https://homes.di.unimi.it/scotti/all/ - """ - - test_len = len(self.data.X_test) - - TP = self.matrix[1][1] - TN = self.matrix[0][0] - FP = self.matrix[0][1] - FN = self.matrix[1][0] - - TPP = (TP * 100)/test_len - FPP = (FP * 100)/test_len - FNP = (FN * 100)/test_len - TNP = (TN * 100)/test_len - - specificity = TN/(TN+FP) - - misc = FP + FN - miscp = (misc * 100)/test_len - - self.helpers.logger.info( - "True Positives: " + str(TP) + "(" + str(TPP) + "%)") - self.helpers.logger.info( - "False Positives: " + str(FP) + "(" + str(FPP) + "%)") - self.helpers.logger.info( - "True Negatives: " + str(TN) + "(" + str(TNP) + "%)") - self.helpers.logger.info( - "False Negatives: " + str(FN) + "(" + str(FNP) + "%)") - - self.helpers.logger.info("Specificity: " + str(specificity)) - self.helpers.logger.info("Misclassification: " + - str(misc) + "(" + str(miscp) + "%)") - - def load(self): - """ Loads the model """ - - with open(self.model_json) as file: - m_json = file.read() - - self.tf_model = tf.keras.models.model_from_json(m_json) - self.tf_model.load_weights(self.weights_file) - - self.helpers.logger.info("Model loaded ") - - self.tf_model.summary() - - def evaluate(self): - """ Evaluates the model """ - - self.predictions() - - metrics = self.tf_model.evaluate( - self.data.X_test, self.data.y_test, verbose=0) - for name, value in zip(self.tf_model.metrics_names, metrics): - self.helpers.logger.info("Metrics: " + name + " " + str(value)) - print() - - self.visualize_metrics() - self.confusion_matrix() - self.figures_of_merit() - exit() - - def predictions(self): - """ Gets a prediction for an image. """ - - self.train_preds = self.tf_model.predict(self.data.X_train) - self.test_preds = self.tf_model.predict(self.data.X_test) - - self.helpers.logger.info("Training predictions: " + str(self.train_preds)) - self.helpers.logger.info("Testing predictions: " + str(self.test_preds)) - - def predict(self, img): - """ Gets a prediction for an image. """ - - predictions = self.tf_model.predict_proba(img) - prediction = np.argmax(predictions, axis=-1) - - return prediction - - def reshape(self, img): - """ Reshapes an image. """ - - dx, dy, dz = img.shape - input_data = img.reshape((-1, dx, dy, dz)) - input_data = input_data / 255.0 - - return input_data - - def test(self): - """ Test mode - - Loops through the test directory and classifies the images. - """ - - files = 0 - tp = 0 - fp = 0 - tn = 0 - fn = 0 - totaltime = 0 - - for testFile in os.listdir(self.testing_dir): - if os.path.splitext(testFile)[1] in self.valid: - files += 1 - fileName = self.testing_dir + "/" + testFile - - start = time.time() - img = cv2.imread(fileName).astype(np.float32) - self.helpers.logger.info("Loaded test image " + fileName) - - img = cv2.resize(img, (self.helpers.confs["data"]["dim"], - self.helpers.confs["data"]["dim"])) - img = self.reshape(img) - - prediction = self.predict(img) - end = time.time() - benchmark = end - start - totaltime += benchmark - - msg = "" - if prediction == 1 and "_1." in testFile: - tp += 1 - msg = "Acute Lymphoblastic Leukemia correctly detected (True Positive) in " + str(benchmark) + " seconds." - elif prediction == 1 and "_0." in testFile: - fp += 1 - msg = "Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in " + str(benchmark) + " seconds." - elif prediction == 0 and "_0." in testFile: - tn += 1 - msg = "Acute Lymphoblastic Leukemia correctly not detected (True Negative) in " + str(benchmark) + " seconds." - elif prediction == 0 and "_1." in testFile: - fn += 1 - msg = "Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in " + str(benchmark) + " seconds." - self.helpers.logger.info(msg) - - self.helpers.logger.info("Images Classified: " + str(files)) - self.helpers.logger.info("True Positives: " + str(tp)) - self.helpers.logger.info("False Positives: " + str(fp)) - self.helpers.logger.info("True Negatives: " + str(tn)) - self.helpers.logger.info("False Negatives: " + str(fn)) - self.helpers.logger.info("Total Time Taken: " + str(totaltime)) - - def test_http(self): - """ HTTP test mode - - Loops through the test directory and classifies the images - by sending data to the classifier using HTTP requests. - """ - - totaltime = 0 - files = 0 - - tp = 0 - fp = 0 - tn = 0 - fn = 0 - - self.addr = "http://" + self.helpers.credentials["server"]["ip"] + \ - ':'+str(self.helpers.credentials["server"]["port"]) + '/Inference' - self.headers = {'content-type': 'image/jpeg'} - - for testFile in os.listdir(self.testing_dir): - if os.path.splitext(testFile)[1] in self.valid: - - start = time.time() - prediction = self.http_request(self.testing_dir + "/" + testFile) - end = time.time() - benchmark = end - start - totaltime += benchmark - - msg = "" - status = "" - outcome = "" - - if prediction["Diagnosis"] == "Positive" and "_1." in testFile: - tp += 1 - status = "correctly" - outcome = "(True Positive)" - elif prediction["Diagnosis"] == "Positive" and "_0." in testFile: - fp += 1 - status = "incorrectly" - outcome = "(False Positive)" - elif prediction["Diagnosis"] == "Negative" and "_0." in testFile: - tn += 1 - status = "correctly" - outcome = "(True Negative)" - elif prediction["Diagnosis"] == "Negative" and "_1." in testFile: - fn += 1 - status = "incorrectly" - outcome = "(False Negative)" - - files += 1 - self.helpers.logger.info("Acute Lymphoblastic Leukemia " + status + - " detected " + outcome + " in " + str(benchmark) + " seconds.") - - self.helpers.logger.info("Images Classified: " + str(files)) - self.helpers.logger.info("True Positives: " + str(tp)) - self.helpers.logger.info("False Positives: " + str(fp)) - self.helpers.logger.info("True Negatives: " + str(tn)) - self.helpers.logger.info("False Negatives: " + str(fn)) - self.helpers.logger.info("Total Time Taken: " + str(totaltime)) + """ Class representing a HIAS AI Model. + + This object represents a HIAS AI Model. HIAS AI + Models are used by AI Agents to process incoming data. + """ + + def prepare_data(self): + """ Creates/sorts dataset. """ + + self.data.remove_testing() + self.data.pre_process() + + self.helpers.logger.info( + "Data preperation complete.") + + def prepare_network(self): + """ Builds the network. + + Replicates the networked outlined in the Acute Leukemia + Classification Using Convolution Neural Network In Clinical + Decision Support System paper using Tensorflow 2.0. + https://airccj.org/CSCP/vol7/csit77505.pdf + """ + + self.val_steps = self.helpers.confs["train"]["val_steps"] + self.batch_size = self.helpers.confs["train"]["batch"] + self.epochs = self.helpers.confs["train"]["epochs"] + + self.tf_model = tf.keras.models.Sequential([ + tf.keras.layers.ZeroPadding2D( + padding=(2, 2), input_shape=self.data.X_train.shape[1:]), + tf.keras.layers.Conv2D(30, (5, 5), strides=1, + padding="valid", activation='relu'), + tf.keras.layers.ZeroPadding2D(padding=(2, 2)), + tf.keras.layers.Conv2D(30, (5, 5), strides=1, + padding="valid", activation='relu'), + tf.keras.layers.MaxPooling2D( + pool_size=(2, 2), strides=2, padding='valid'), + tf.keras.layers.Flatten(), + tf.keras.layers.Dense(2), + tf.keras.layers.Activation('softmax') + ], + "AllOneApiClassifier") + self.tf_model.summary() + + self.helpers.logger.info( + "Network initialization complete.") + + def train(self): + """ Trains the model + + Trains the neural network. + """ + + self.helpers.logger.info("Using Adam Optimizer.") + optimizer = tf.keras.optimizers.Adam( + lr=self.helpers.confs["train"]["learning_rate_adam"], + decay = self.helpers.confs["train"]["decay_adam"]) + + self.tf_model.compile( + optimizer=optimizer, loss='binary_crossentropy', + metrics=[tf.keras.metrics.BinaryAccuracy(name='acc'), + tf.keras.metrics.Precision(name='precision'), + tf.keras.metrics.Recall(name='recall'), + tf.keras.metrics.AUC(name='auc') ]) + + self.history = self.tf_model.fit( + self.data.X_train, self.data.y_train, + validation_data=(self.data.X_test, self.data.y_test), + validation_steps=self.val_steps, + epochs=self.epochs) + + print(self.history) + print("") + + self.freeze_model() + self.save_model_as_json() + self.save_weights() + + def freeze_model(self): + """ Freezes the model """ + + tf.saved_model.save( + self.tf_model, self.helpers.confs["model"]["saved_model_dir"]) + + fmodel = tf.function(lambda x: self.tf_model(x)) + fmodel = fmodel.get_concrete_function( + x=tf.TensorSpec( + self.tf_model.inputs[0].shape, self.tf_model.inputs[0].dtype)) + + freeze = convert_variables_to_constants_v2(fmodel) + freeze.graph.as_graph_def() + + layers = [op.name for op in freeze.graph.get_operations()] + self.helpers.logger.info("Frozen model layers") + for layer in layers: + self.helpers.logger.info(layer) + + self.helpers.logger.info("Frozen model inputs") + self.helpers.logger.info(freeze.inputs) + self.helpers.logger.info("Frozen model outputs") + self.helpers.logger.info(freeze.outputs) + + tf.io.write_graph(graph_or_graph_def=freeze.graph, + logdir=self.helpers.confs["model"]["freezing_log_dir"], + name=self.helpers.confs["model"]["frozen"], + as_text=False) + + def save_model_as_json(self): + """ Saves the model as JSON """ + + with open(self.model_json, "w") as file: + file.write(self.tf_model.to_json()) + + self.helpers.logger.info( + "Model JSON saved " + self.model_json) + + def save_weights(self): + """ Saves the model weights """ + + self.tf_model.save_weights(self.weights_file) + self.helpers.logger.info( + "Weights saved " + self.weights_file) + + def visualize_metrics(self): + """ Visualize the metrics. """ + + plt.plot(self.history.history['acc']) + plt.plot(self.history.history['val_acc']) + plt.title('Model Accuracy') + plt.ylabel('Accuracy') + plt.xlabel('Epoch') + plt.ylim((0, 1)) + plt.legend(['Train', 'Validate'], loc='upper left') + plt.savefig('model/plots/accuracy.png') + plt.show() + plt.clf() + + plt.plot(self.history.history['loss']) + plt.plot(self.history.history['val_loss']) + plt.title('Model Loss') + plt.ylabel('loss') + plt.xlabel('Epoch') + plt.legend(['Train', 'Validate'], loc='upper left') + plt.savefig('model/plots/loss.png') + plt.show() + plt.clf() + + plt.plot(self.history.history['auc']) + plt.plot(self.history.history['val_auc']) + plt.title('Model AUC') + plt.ylabel('AUC') + plt.xlabel('Epoch') + plt.legend(['Train', 'Validate'], loc='upper left') + plt.savefig('model/plots/auc.png') + plt.show() + plt.clf() + + plt.plot(self.history.history['precision']) + plt.plot(self.history.history['val_precision']) + plt.title('Model Precision') + plt.ylabel('Precision') + plt.xlabel('Epoch') + plt.legend(['Train', 'Validate'], loc='upper left') + plt.savefig('model/plots/precision.png') + plt.show() + plt.clf() + + plt.plot(self.history.history['recall']) + plt.plot(self.history.history['val_recall']) + plt.title('Model Recall') + plt.ylabel('Recall') + plt.xlabel('Epoch') + plt.legend(['Train', 'Validate'], loc='upper left') + plt.savefig('model/plots/recall.png') + plt.show() + plt.clf() + + def confusion_matrix(self): + """ Prints/displays the confusion matrix. """ + + self.matrix = confusion_matrix( + self.data.y_test.argmax(axis=1), + self.test_preds.argmax(axis=1)) + + self.helpers.logger.info( + "Confusion Matrix: " + str(self.matrix)) + print("") + + plot_confusion_matrix(conf_mat=self.matrix) + plt.savefig('model/plots/confusion-matrix.png') + plt.show() + plt.clf() + + def figures_of_merit(self): + """ Calculates/prints the figures of merit. + + https://homes.di.unimi.it/scotti/all/ + """ + + test_len = len(self.data.X_test) + + TP = self.matrix[1][1] + TN = self.matrix[0][0] + FP = self.matrix[0][1] + FN = self.matrix[1][0] + + TPP = (TP * 100)/test_len + FPP = (FP * 100)/test_len + FNP = (FN * 100)/test_len + TNP = (TN * 100)/test_len + + specificity = TN/(TN+FP) + + misc = FP + FN + miscp = (misc * 100)/test_len + + self.helpers.logger.info( + "True Positives: " + str(TP) + "(" + str(TPP) + "%)") + self.helpers.logger.info( + "False Positives: " + str(FP) + "(" + str(FPP) + "%)") + self.helpers.logger.info( + "True Negatives: " + str(TN) + "(" + str(TNP) + "%)") + self.helpers.logger.info( + "False Negatives: " + str(FN) + "(" + str(FNP) + "%)") + + self.helpers.logger.info( + "Specificity: " + str(specificity)) + self.helpers.logger.info( + "Misclassification: " + str(misc) + "(" + str(miscp) + "%)") + + def load(self): + """ Loads the model """ + + with open(self.model_json) as file: + m_json = file.read() + + self.tf_model = tf.keras.models.model_from_json(m_json) + self.tf_model.load_weights(self.weights_file) + + self.helpers.logger.info("Model loaded ") + + self.tf_model.summary() + + def evaluate(self): + """ Evaluates the model """ + + self.predictions() + + metrics = self.tf_model.evaluate( + self.data.X_test, self.data.y_test, verbose=0) + for name, value in zip(self.tf_model.metrics_names, metrics): + self.helpers.logger.info("Metrics: " + name + " " + str(value)) + print() + + self.visualize_metrics() + self.confusion_matrix() + self.figures_of_merit() + exit() + + def predictions(self): + """ Gets a prediction for an image. """ + + self.train_preds = self.tf_model.predict(self.data.X_train) + self.test_preds = self.tf_model.predict(self.data.X_test) + + self.helpers.logger.info( + "Training predictions: " + str(self.train_preds)) + self.helpers.logger.info( + "Testing predictions: " + str(self.test_preds)) + + def predict(self, img): + """ Gets a prediction for an image. """ + + predictions = self.tf_model.predict_proba(img) + prediction = np.argmax(predictions, axis=-1) + + return prediction + + def reshape(self, img): + """ Reshapes an image. """ + + dx, dy, dz = img.shape + input_data = img.reshape((-1, dx, dy, dz)) + input_data = input_data / 255.0 + + return input_data + + def test(self): + """ Test mode + + Loops through the test directory and classifies the images. + """ + + files = 0 + tp = 0 + fp = 0 + tn = 0 + fn = 0 + totaltime = 0 + + for testFile in os.listdir(self.testing_dir): + if os.path.splitext(testFile)[1] in self.valid: + files += 1 + fileName = self.testing_dir + "/" + testFile + + start = time.time() + img = cv2.imread(fileName).astype(np.float32) + self.helpers.logger.info( + "Loaded test image " + fileName) + + img = cv2.resize(img, (self.helpers.confs["data"]["dim"], + self.helpers.confs["data"]["dim"])) + img = self.reshape(img) + + prediction = self.predict(img) + end = time.time() + benchmark = end - start + totaltime += benchmark + + msg = "" + if prediction == 1 and "_1." in testFile: + tp += 1 + msg = "Acute Lymphoblastic Leukemia correctly detected (True Positive) in " \ + + str(benchmark) + " seconds." + elif prediction == 1 and "_0." in testFile: + fp += 1 + msg = "Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in " \ + + str(benchmark) + " seconds." + elif prediction == 0 and "_0." in testFile: + tn += 1 + msg = "Acute Lymphoblastic Leukemia correctly not detected (True Negative) in " \ + + str(benchmark) + " seconds." + elif prediction == 0 and "_1." in testFile: + fn += 1 + msg = "Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in " \ + + str(benchmark) + " seconds." + self.helpers.logger.info(msg) + + self.helpers.logger.info( + "Images Classified: " + str(files)) + self.helpers.logger.info( + "True Positives: " + str(tp)) + self.helpers.logger.info( + "False Positives: " + str(fp)) + self.helpers.logger.info( + "True Negatives: " + str(tn)) + self.helpers.logger.info( + "False Negatives: " + str(fn)) + self.helpers.logger.info( + "Total Time Taken: " + str(totaltime)) + + def test_http(self): + """ HTTP test mode + + Loops through the test directory and classifies the images + by sending data to the classifier using HTTP requests. + """ + + totaltime = 0 + files = 0 + + tp = 0 + fp = 0 + tn = 0 + fn = 0 + + self.addr = "http://" + self.helpers.get_ip_addr() + \ + ':'+str(self.helpers.confs["agent"]["port"]) + '/Inference' + self.headers = {'content-type': 'image/jpeg'} + + for testFile in os.listdir(self.testing_dir): + if os.path.splitext(testFile)[1] in self.valid: + + start = time.time() + prediction = self.http_request( + self.testing_dir + "/" + testFile) + end = time.time() + benchmark = end - start + totaltime += benchmark + + msg = "" + status = "" + outcome = "" + + if prediction["Diagnosis"] == "Positive" and "_1." in testFile: + tp += 1 + status = "correctly" + outcome = "(True Positive)" + elif prediction["Diagnosis"] == "Positive" and "_0." in testFile: + fp += 1 + status = "incorrectly" + outcome = "(False Positive)" + elif prediction["Diagnosis"] == "Negative" and "_0." in testFile: + tn += 1 + status = "correctly" + outcome = "(True Negative)" + elif prediction["Diagnosis"] == "Negative" and "_1." in testFile: + fn += 1 + status = "incorrectly" + outcome = "(False Negative)" + + files += 1 + self.helpers.logger.info( + "Acute Lymphoblastic Leukemia " + status + + " detected " + outcome + " in " + str(benchmark) + " seconds.") + + self.helpers.logger.info( + "Images Classified: " + str(files)) + self.helpers.logger.info( + "True Positives: " + str(tp)) + self.helpers.logger.info( + "False Positives: " + str(fp)) + self.helpers.logger.info( + "True Negatives: " + str(tn)) + self.helpers.logger.info( + "False Negatives: " + str(fn)) + self.helpers.logger.info( + "Total Time Taken: " + str(totaltime)) diff --git a/modules/model_openvino.py b/modules/model_openvino.py index 4093290..59a0f0d 100644 --- a/modules/model_openvino.py +++ b/modules/model_openvino.py @@ -1,8 +1,13 @@ #!/usr/bin/env python """ Abstract class representing a HIAS AI OpenVINO Model. -Represents a HIAS AI OpenVINO Model. HIAS AI OpenVINO Models are used by AI Agents to process -incoming data. +Represents a HIAS AI OpenVINO Model. HIAS AI OpenVINO +Models are used by AI Agents to process incoming data. + +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal @@ -23,7 +28,7 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-3 +- Adam Milton-Barker """ @@ -37,166 +42,187 @@ class model_openvino(AbstractOpenVINO): - """ Class representing a HIAS AI OpenVINO Model. - - This object represents a HIAS AI OpenVINO Model. HIAS AI OpenVINO Models are used by AI Agents - to process incoming data. - """ - - def load(self): - """ Loads the model """ - - mxml = self.helpers.confs["rpi4"]["ir"] - mbin = os.path.splitext(mxml)[0] + ".bin" - - self.net = cv2.dnn.readNet(mxml, mbin) - self.net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD) - - self.helpers.logger.info("OpenVINO loaded.") - - def setBlob(self, frame): - """ Gets a blob from the color frame """ - - blob = cv2.dnn.blobFromImage(frame, self.helpers.confs["rpi4"]["inScaleFactor"], - size=(self.imsize, self.imsize), - mean=(self.helpers.confs["rpi4"]["meanVal"], - self.helpers.confs["rpi4"]["meanVal"], - self.helpers.confs["rpi4"]["meanVal"]), - swapRB=True, crop=False) - - self.net.setInput(blob) - - def forwardPass(self): - """ Gets a blob from the color frame """ - - out = self.net.forward() - - return out - - def predict(self): - """ Gets a prediction for an image. """ - - predictions = self.forwardPass() - predictions = predictions[0] - idx = np.argsort(predictions)[::-1][0] - prediction = self.helpers.confs["data"]["labels"][idx] - - return prediction - - def test(self): - """ Test mode - - Loops through the test directory and classifies the images. - """ - - files = 0 - tp = 0 - fp = 0 - tn = 0 - fn = 0 - totaltime = 0 - - for testFile in os.listdir(self.testing_dir): - if os.path.splitext(testFile)[1] in self.valid: - - files += 1 - fileName = self.testing_dir + "/" + testFile - - img = cv2.imread(fileName) - self.helpers.logger.info("Loaded test image " + fileName) - self.setBlob(self.resize(img)) - start = time.time() - prediction = self.predict() - end = time.time() - benchmark = end - start - totaltime += benchmark - - msg = "" - if prediction == 1 and "_1." in testFile: - tp += 1 - msg = "Acute Lymphoblastic Leukemia correctly detected (True Positive) in " + str(benchmark) + " seconds." - elif prediction == 1 and "_0." in testFile: - fp += 1 - msg = "Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in " + str(benchmark) + " seconds." - elif prediction == 0 and "_0." in testFile: - tn += 1 - msg = "Acute Lymphoblastic Leukemia correctly not detected (True Negative) in " + str(benchmark) + " seconds." - elif prediction == 0 and "_1." in testFile: - fn += 1 - msg = "Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in " + str(benchmark) + " seconds." - self.helpers.logger.info(msg) - - self.helpers.logger.info("Images Classifier: " + str(files)) - self.helpers.logger.info("True Positives: " + str(tp)) - self.helpers.logger.info("False Positives: " + str(fp)) - self.helpers.logger.info("True Negatives: " + str(tn)) - self.helpers.logger.info("False Negatives: " + str(fn)) - self.helpers.logger.info("Total Time Taken: " + str(totaltime)) - - def test_http(self): - """ HTTP test mode - - Loops through the test directory and classifies the images - by sending data to the classifier using HTTP requests. - """ - - totaltime = 0 - files = 0 - - tp = 0 - fp = 0 - tn = 0 - fn = 0 - - self.addr = "http://" + self.helpers.credentials["server"]["ip"] + \ - ':'+str(self.helpers.credentials["server"]["port"]) + '/Inference' - self.headers = {'content-type': 'image/jpeg'} - - for testFile in os.listdir(self.testing_dir): - if os.path.splitext(testFile)[1] in self.valid: - - start = time.time() - prediction = self.http_request(self.testing_dir + "/" + testFile) - end = time.time() - benchmark = end - start - totaltime += benchmark - - msg = "" - status = "" - outcome = "" - - if prediction["Diagnosis"] == "Positive" and "_1." in testFile: - tp += 1 - status = "correctly" - outcome = "(True Positive)" - elif prediction["Diagnosis"] == "Positive" and "_0." in testFile: - fp += 1 - status = "incorrectly" - outcome = "(False Positive)" - elif prediction["Diagnosis"] == "Negative" and "_0." in testFile: - tn += 1 - status = "correctly" - outcome = "(True Negative)" - elif prediction["Diagnosis"] == "Negative" and "_1." in testFile: - fn += 1 - status = "incorrectly" - outcome = "(False Negative)" - - files += 1 - self.helpers.logger.info("Acute Lymphoblastic Leukemia " + status + - " detected " + outcome + " in " + str(benchmark) + " seconds.") - - self.helpers.logger.info("Images Classified: " + str(files)) - self.helpers.logger.info("True Positives: " + str(tp)) - self.helpers.logger.info("False Positives: " + str(fp)) - self.helpers.logger.info("True Negatives: " + str(tn)) - self.helpers.logger.info("False Negatives: " + str(fn)) - self.helpers.logger.info("Total Time Taken: " + str(totaltime)) - - def resize(self, img): - """ Reshapes an image. """ - - img = cv2.resize(img, (self.helpers.confs["data"]["dim"], + """ Class representing a HIAS AI OpenVINO Model. + + This object represents a HIAS AI OpenVINO Model. + HIAS AI OpenVINO Models are used by AI Agents + to process incoming data. + """ + + def load(self): + """ Loads the model """ + + mxml = self.helpers.confs["rpi4"]["ir"] + mbin = os.path.splitext(mxml)[0] + ".bin" + + self.net = cv2.dnn.readNet(mxml, mbin) + self.net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD) + + self.helpers.logger.info("OpenVINO loaded.") + + def setBlob(self, frame): + """ Gets a blob from the color frame """ + + blob = cv2.dnn.blobFromImage( + frame, self.helpers.confs["rpi4"]["inScaleFactor"], + size=(self.imsize, self.imsize), + mean=(self.helpers.confs["rpi4"]["meanVal"], + self.helpers.confs["rpi4"]["meanVal"], + self.helpers.confs["rpi4"]["meanVal"]), + swapRB=True, crop=False) + + self.net.setInput(blob) + + def forwardPass(self): + """ Gets a blob from the color frame """ + + out = self.net.forward() + + return out + + def predict(self): + """ Gets a prediction for an image. """ + + predictions = self.forwardPass() + predictions = predictions[0] + idx = np.argsort(predictions)[::-1][0] + prediction = self.helpers.confs["data"]["labels"][idx] + + return prediction + + def test(self): + """ Test mode + + Loops through the test directory and classifies the images. + """ + + files = 0 + tp = 0 + fp = 0 + tn = 0 + fn = 0 + totaltime = 0 + + for testFile in os.listdir(self.testing_dir): + if os.path.splitext(testFile)[1] in self.valid: + + files += 1 + fileName = self.testing_dir + "/" + testFile + + img = cv2.imread(fileName) + self.helpers.logger.info( + "Loaded test image " + fileName) + self.setBlob(self.resize(img)) + start = time.time() + prediction = self.predict() + end = time.time() + benchmark = end - start + totaltime += benchmark + + msg = "" + if prediction == 1 and "_1." in testFile: + tp += 1 + msg = "Acute Lymphoblastic Leukemia correctly detected (True Positive) in " \ + + str(benchmark) + " seconds." + elif prediction == 1 and "_0." in testFile: + fp += 1 + msg = "Acute Lymphoblastic Leukemia incorrectly detected (False Positive) in " \ + + str(benchmark) + " seconds." + elif prediction == 0 and "_0." in testFile: + tn += 1 + msg = "Acute Lymphoblastic Leukemia correctly not detected (True Negative) in " \ + + str(benchmark) + " seconds." + elif prediction == 0 and "_1." in testFile: + fn += 1 + msg = "Acute Lymphoblastic Leukemia incorrectly not detected (False Negative) in " \ + + str(benchmark) + " seconds." + self.helpers.logger.info(msg) + + self.helpers.logger.info( + "Images Classifier: " + str(files)) + self.helpers.logger.info( + "True Positives: " + str(tp)) + self.helpers.logger.info( + "False Positives: " + str(fp)) + self.helpers.logger.info( + "True Negatives: " + str(tn)) + self.helpers.logger.info( + "False Negatives: " + str(fn)) + self.helpers.logger.info( + "Total Time Taken: " + str(totaltime)) + + def test_http(self): + """ HTTP test mode + + Loops through the test directory and classifies the images + by sending data to the classifier using HTTP requests. + """ + + totaltime = 0 + files = 0 + + tp = 0 + fp = 0 + tn = 0 + fn = 0 + + self.addr = "http://" + self.helpers.get_ip_addr() + \ + ':'+str(self.helpers.credentials["server"]["port"]) + '/Inference' + self.headers = {'content-type': 'image/jpeg'} + + for testFile in os.listdir(self.testing_dir): + if os.path.splitext(testFile)[1] in self.valid: + + start = time.time() + prediction = self.http_request( + self.testing_dir + "/" + testFile) + end = time.time() + benchmark = end - start + totaltime += benchmark + + msg = "" + status = "" + outcome = "" + + if prediction["Diagnosis"] == "Positive" and "_1." in testFile: + tp += 1 + status = "correctly" + outcome = "(True Positive)" + elif prediction["Diagnosis"] == "Positive" and "_0." in testFile: + fp += 1 + status = "incorrectly" + outcome = "(False Positive)" + elif prediction["Diagnosis"] == "Negative" and "_0." in testFile: + tn += 1 + status = "correctly" + outcome = "(True Negative)" + elif prediction["Diagnosis"] == "Negative" and "_1." in testFile: + fn += 1 + status = "incorrectly" + outcome = "(False Negative)" + + files += 1 + self.helpers.logger.info( + "Acute Lymphoblastic Leukemia " + status + + " detected " + outcome + " in " + str(benchmark) + " seconds.") + + self.helpers.logger.info( + "Images Classified: " + str(files)) + self.helpers.logger.info( + "True Positives: " + str(tp)) + self.helpers.logger.info( + "False Positives: " + str(fp)) + self.helpers.logger.info( + "True Negatives: " + str(tn)) + self.helpers.logger.info( + "False Negatives: " + str(fn)) + self.helpers.logger.info( + "Total Time Taken: " + str(totaltime)) + + def resize(self, img): + """ Reshapes an image. """ + + img = cv2.resize(img, (self.helpers.confs["data"]["dim"], self.helpers.confs["data"]["dim"])) - return img + return img diff --git a/modules/mqtt.py b/modules/mqtt.py index 9d25c40..4eaca2f 100644 --- a/modules/mqtt.py +++ b/modules/mqtt.py @@ -4,6 +4,11 @@ This module connects devices, applications, robots and other softwares to the HIAS iotJumpWay MQTT Broker. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -30,134 +35,155 @@ class mqtt(): - """iotJumpWay MQTT connection.""" - - def __init__(self, - helpers, - client_type, - configs): - """ Initializes the class. """ - - self.configs = configs - self.client_type = client_type - - self.helpers = helpers - self.program = "HIAS iotJumpWay MQTT module" - - self.mqtt_config = {} - self.module_topics = {} - - self.agent = [ - 'host', - 'port', - 'location', - 'zone', - 'entity', - 'name', - 'un', - 'up' - ] - - self.helpers.logger.info(self.program + " initialization complete.") - - def configure(self): - - # Checks required parameters - self.client_id = self.configs['name'] - for param in self.agent: - if self.configs[param] is None: - raise ConfigurationException(param + " parameter is required!") - - # Sets MQTT connection configuration - self.mqtt_config["tls"] = "/etc/ssl/certs/DST_Root_CA_X3.pem" - self.mqtt_config["host"] = self.configs['host'] - self.mqtt_config["port"] = self.configs['port'] - - # Sets MQTT topics - self.module_topics["statusTopic"] = '%s/Agents/%s/%s/Status' % ( - self.configs['location'], self.configs['zone'], self.configs['entity']) - - # Sets MQTT callbacks - self.commandsCallback = None - - self.helpers.logger.info( - "iotJumpWay " + self.client_type + " connection configured.") - - def start(self): - - self.mClient = pmqtt.Client(client_id=self.client_id, clean_session=True) - self.mClient.will_set(self.module_topics["statusTopic"], "OFFLINE", 0, False) - self.mClient.tls_set(self.mqtt_config["tls"], certfile=None, keyfile=None) - self.mClient.on_connect = self.on_connect - self.mClient.on_message = self.on_message - self.mClient.on_publish = self.on_publish - self.mClient.on_subscribe = self.on_subscribe - self.mClient.username_pw_set( - str(self.configs['un']), str(self.configs['up'])) - self.mClient.connect(self.mqtt_config["host"], self.mqtt_config["port"], 10) - self.mClient.loop_start() - - self.helpers.logger.info( - "iotJumpWay " + self.client_type + " connection started.") - - def on_connect(self, client, obj, flags, rc): - - self.helpers.logger.info( - "iotJumpWay " + self.client_type + " connection successful.") - self.helpers.logger.info("rc: " + str(rc)) - - self.statusPublish("ONLINE") - - def statusPublish(self, data): - - self.mClient.publish(self.module_topics["statusTopic"], data) - self.helpers.logger.info("Published to " + self.client_type + " status.") - - def on_subscribe(self, client, obj, mid, granted_qos): - - self.helpers.logger.info("iotJumpWay " + self.client_type + " subscription") - - def on_message(self, client, obj, msg): - - splitTopic = msg.topic.split("/") - connType = splitTopic[1] - - topic = splitTopic[4] - - self.helpers.logger.info(msg.payload) - self.helpers.logger.info("iotJumpWay " + connType + - " " + topic + " communication received.") - - if topic == 'Commands': - if self.commandsCallback == None: - self.helpers.logger.info( - connType + " comands callback required (commandsCallback) !") - else: - self.commandsCallback(msg.topic, msg.payload) - - def publish(self, channel, data): - - channel = '%s/Agents/%s/%s/%s' % (self.configs['location'], - self.configs['zone'], self.configs['entity'], channel) - - self.mClient.publish(channel, json.dumps(data)) - self.helpers.logger.info("Published to " + channel) - return True - - def subscribe(self, application=None, channelID=None, qos=0): - - channel = '%s/#' % (self.configs['location']) - self.mClient.subscribe(channel, qos=qos) - self.helpers.logger.info("-- Agent subscribed to all channels") - return True - - def on_publish(self, client, obj, mid): - self.helpers.logger.info("Published: "+str(mid)) - - def on_log(self, client, obj, level, string): - print(string) - - def disconnect(self): - self.statusPublish("OFFLINE") - self.mClient.disconnect() - self.mClient.loop_stop() + """iotJumpWay MQTT connection.""" + + def __init__(self, + helpers, + client_type, + configs): + """ Initializes the class. """ + + self.configs = configs + self.client_type = client_type + + self.helpers = helpers + self.program = "HIAS iotJumpWay MQTT module" + + self.mqtt_config = {} + self.module_topics = {} + + self.agent = [ + 'host', + 'port', + 'location', + 'zone', + 'entity', + 'name', + 'un', + 'up' + ] + + self.helpers.logger.info( + self.program + " initialization complete.") + + def configure(self): + + # Checks required parameters + self.client_id = self.configs['name'] + for param in self.agent: + if self.configs[param] is None: + raise ConfigurationException( + param + " parameter is required!") + + # Sets MQTT connection configuration + self.mqtt_config["tls"] = "/etc/ssl/certs/DST_Root_CA_X3.pem" + self.mqtt_config["host"] = self.configs['host'] + self.mqtt_config["port"] = self.configs['port'] + + # Sets MQTT topics + self.module_topics["statusTopic"] = '%s/Agents/%s/%s/Status' % ( + self.configs['location'], self.configs['zone'], + self.configs['entity']) + + # Sets MQTT callbacks + self.commandsCallback = None + + self.helpers.logger.info( + "iotJumpWay " + self.client_type + " connection configured.") + + def start(self): + + self.mqtt_client = pmqtt.Client( + client_id=self.client_id, clean_session=True) + self.mqtt_client.will_set( + self.module_topics["statusTopic"], + "OFFLINE", 0, False) + self.mqtt_client.tls_set( + self.mqtt_config["tls"], + certfile=None, + keyfile=None) + self.mqtt_client.on_connect = self.on_connect + self.mqtt_client.on_message = self.on_message + self.mqtt_client.on_publish = self.on_publish + self.mqtt_client.on_subscribe = self.on_subscribe + self.mqtt_client.username_pw_set( + str(self.configs['un']), + str(self.configs['up'])) + self.mqtt_client.connect( + self.mqtt_config["host"], + self.mqtt_config["port"], 10) + self.mqtt_client.loop_start() + + self.helpers.logger.info( + "iotJumpWay " + self.client_type \ + + " connection started.") + + def on_connect(self, client, obj, flags, rc): + + self.helpers.logger.info( + "iotJumpWay " + self.client_type \ + + " connection successful.") + self.helpers.logger.info("rc: " + str(rc)) + + self.status_publish("ONLINE") + + def status_publish(self, data): + + self.mqtt_client.publish( + self.module_topics["statusTopic"], data) + self.helpers.logger.info( + "Published to " + self.client_type + " status.") + + def on_subscribe(self, client, obj, mid, granted_qos): + + self.helpers.logger.info( + "iotJumpWay " + self.client_type + " subscription") + + def on_message(self, client, obj, msg): + + splitTopic = msg.topic.split("/") + connType = splitTopic[1] + + topic = splitTopic[4] + + self.helpers.logger.info(msg.payload) + self.helpers.logger.info( + "iotJumpWay " + connType + " " \ + + topic + " communication received.") + + if topic == 'Commands': + if self.commandsCallback == None: + self.helpers.logger.info( + connType + " comands callback required (commandsCallback) !") + else: + self.commandsCallback(msg.topic, msg.payload) + + def publish(self, channel, data): + + channel = '%s/Agents/%s/%s/%s' % (self.configs['location'], + self.configs['zone'], self.configs['entity'], channel) + + self.mqtt_client.publish(channel, json.dumps(data)) + self.helpers.logger.info("Published to " + channel) + return True + + def subscribe(self, application=None, channelID=None, qos=0): + + channel = '%s/Agents/%s/%s/#' % (self.configs['location'], + self.configs['zone'], self.configs['entity']) + self.mqtt_client.subscribe(channel, qos=qos) + self.helpers.logger.info( + "Agent subscribed to all channels") + return True + + def on_publish(self, client, obj, mid): + self.helpers.logger.info( + "Published: "+str(mid)) + + def on_log(self, client, obj, level, string): + print(string) + + def disconnect(self): + self.status_publish("OFFLINE") + self.mqtt_client.disconnect() + self.mqtt_client.loop_stop() diff --git a/modules/server.py b/modules/server.py index 134f24a..4b41304 100644 --- a/modules/server.py +++ b/modules/server.py @@ -3,6 +3,11 @@ Class for the HIAS IoT Agent server/API. +MIT License + +Copyright (c) 2021 Asociación de Investigacion en Inteligencia Artificial +Para la Leucemia Peter Moss + Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights @@ -22,16 +27,13 @@ SOFTWARE. Contributors: -- Adam Milton-Barker - First version - 2021-5-2 +- Adam Milton-Barker """ import cv2 import json import jsonpickle -import os -import requests -import time import numpy as np @@ -42,81 +44,83 @@ from flask import Flask, request, Response class server(AbstractServer): - """ Server/API class. + """ Server/API class. - Class for the HIAS IoT Agent server/API. - """ + Class for the HIAS IoT Agent server/API. + """ - def predict(self, req): - """ Classifies an image sent via HTTP. """ + def predict(self, req): + """ Classifies an image sent via HTTP. """ - if len(req.files) != 0: - img = Image.open(req.files['file'].stream) - else: - img = Image.open(BytesIO(req.data)) + img = np.fromstring(req.data, np.uint8) + img = cv2.imdecode(img, cv2.IMREAD_COLOR) - img = self.model.data.reshape(img) + img = cv2.resize(img, (self.model.data.dim, + self.model.data.dim)) + img = self.model.reshape(img) - return self.model.predict(img) + return self.model.predict(img) - def predict_openvino(self, req): - """ Classifies an image sent via HTTP using OpenVINO. """ + def predict_openvino(self, req): + """ Classifies an image sent via HTTP using OpenVINO. """ - if len(req.files) != 0: - img = np.fromstring(req.files['file'].read(), np.uint8) - else: - img = np.fromstring(req.data, np.uint8) + if len(req.files) != 0: + img = np.fromstring(req.files['file'].read(), np.uint8) + else: + img = np.fromstring(req.data, np.uint8) - img = cv2.imdecode(img, cv2.IMREAD_UNCHANGED) + img = cv2.imdecode(img, cv2.IMREAD_UNCHANGED) - img = self.model.resize(img) - self.model.setBlob(img) + img = self.model.resize(img) + self.model.setBlob(img) - return self.model.predict() + return self.model.predict() - def start(self): - """ Starts the server. """ + def start(self): + """ Starts the server. """ - app = Flask(self.helpers.credentials["iotJumpWay"]["name"]) + app = Flask(self.helpers.credentials["iotJumpWay"]["name"]) - @app.route('/Inference', methods=['POST']) - def Inference(): - """ Responds to HTTP POST requests. """ + @app.route('/Inference', methods=['POST']) + def Inference(): + """ Responds to HTTP POST requests. """ - self.mqtt.publish("States", { - "Type": "Prediction", - "Name": self.helpers.credentials["iotJumpWay"]["name"], - "State": "Processing", - "Message": "Processing data" - }) + self.mqtt.publish("States", { + "Type": "Prediction", + "Name": self.helpers.credentials["iotJumpWay"]["name"], + "State": "Processing", + "Message": "Processing data" + }) - message = "" - if self.model_type == "CNN": - prediction = self.predict(request) - else: - prediction = self.predict_openvino(request) + message = "" + if self.model_type == "CNN": + prediction = self.predict(request) + else: + prediction = self.predict_openvino(request) - if prediction == 1: - message = "Acute Lymphoblastic Leukemia detected!" - diagnosis = "Positive" - elif prediction == 0: - message = "Acute Lymphoblastic Leukemia not detected!" - diagnosis = "Negative" + if prediction == 1: + message = "Acute Lymphoblastic Leukemia detected!" + diagnosis = "Positive" + elif prediction == 0: + message = "Acute Lymphoblastic Leukemia not detected!" + diagnosis = "Negative" - self.mqtt.publish("States", { - "Type": "Prediction", - "Name": self.helpers.credentials["iotJumpWay"]["name"], - "State": diagnosis, - "Message": message - }) + self.mqtt.publish("States", { + "Type": "Prediction", + "Name": self.helpers.credentials["iotJumpWay"]["name"], + "State": diagnosis, + "Message": message + }) - resp = jsonpickle.encode({ - 'Response': 'OK', - 'Message': message, - 'Diagnosis': diagnosis - }) + resp = jsonpickle.encode({ + 'Response': 'OK', + 'Message': message, + 'Diagnosis': diagnosis + }) - return Response(response=resp, status=200, mimetype="application/json") + return Response( + response=resp, status=200, mimetype="application/json") - app.run(host=self.helpers.credentials["server"]["ip"], - port=self.helpers.credentials["server"]["port"]) + app.run( + host=self.helpers.get_ip_addr(), + port=self.helpers.credentials["server"]["port"]) \ No newline at end of file diff --git a/scripts/install_rpi4.sh b/scripts/install-rpi4.sh similarity index 58% rename from scripts/install_rpi4.sh rename to scripts/install-rpi4.sh index 20c9a3d..9f87860 100644 --- a/scripts/install_rpi4.sh +++ b/scripts/install-rpi4.sh @@ -1,13 +1,12 @@ #!/bin/bash -FMSG="- Acute Lymphoblastic Leukemia oneAPI Classifier installation terminated" +FMSG="HIAS ALL oneAPI Classifier installation terminated!" -read -p "? This script will install the required Python libraries. Are you ready (y/n)? " cmsg - -if [ "$cmsg" = "Y" -o "$cmsg" = "y" ]; then - - echo "- Installing required Python libraries" +printf -- 'This script will install HIAS ALL oneAPI Classifier on your Raspberry Pi.\n'; +printf -- '\033[33m WARNING: This is an inteteractive installation, please follow instructions provided. \033[0m\n'; +read -p "Proceed (y/n)? " proceed +if [ "$proceed" = "Y" -o "$proceed" = "y" ]; then sudo apt update sudo apt upgrade sudo apt install -y libgtk-3-dev @@ -26,10 +25,8 @@ if [ "$cmsg" = "Y" -o "$cmsg" = "y" ]; then sudo pip3 install jsonpickle sudo pip3 install paho-mqtt sudo pip3 install pybind11 - #wget https://github.com/Qengineering/Tensorflow-Raspberry-Pi/raw/master/tensorflow-2.1.0-cp37-cp37m-linux_armv7l.whl - #sudo -H pip3 install tensorflow-2.1.0-cp37-cp37m-linux_armv7l.whl - + printf -- '\033[32m SUCCESS: Congratulations! HIAS ALL oneAPI Classifier installed successfully! \033[0m\n'; else echo $FMSG; - exit + exit 1 fi \ No newline at end of file diff --git a/scripts/install.sh b/scripts/install.sh index fa8991d..e1e5427 100644 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -1,16 +1,29 @@ #!/bin/bash -echo "-- Installing requirements" -echo " " -conda install opencv -conda install psutil -conda install requests -conda install flask -conda install matplotlib -conda install tornado -conda install Pillow -conda install jsonpickle -conda install scikit-learn -conda install scikit-image -pip install mlxtend -echo "-- Done" \ No newline at end of file +FMSG="HIAS ALL oneAPI Classifier installation terminated!" + +printf -- 'This script will install HIAS ALL oneAPI Classifier on your Ubuntu development machine.\n'; +printf -- '\033[33m WARNING: This is an inteteractive installation, please follow instructions provided. \033[0m\n'; + +read -p "Proceed (y/n)? " proceed +if [ "$proceed" = "Y" -o "$proceed" = "y" ]; then + conda install flask + conda install jsonpickle + conda install jupyter + conda install matplotlib + conda install nb_conda + conda install opencv + conda install -c conda-forge paho-mqtt + conda install Pillow + conda install psutil + conda install requests + conda install scikit-learn + conda install scikit-learn-intelex + conda install scikit-image + conda install tornado + pip install mlxtend + printf -- '\033[32m SUCCESS: Congratulations! HIAS ALL oneAPI Classifier installed successfully! \033[0m\n'; +else + echo $FMSG; + exit 1 +fi \ No newline at end of file