Skip to content

Latest commit

 

History

History
306 lines (222 loc) · 12 KB

README.md

File metadata and controls

306 lines (222 loc) · 12 KB

Jespipe

An easy-to-use system for conducting adversarial machine learning analysis using OpenMPI, XML control files, and user-written plugins.
Explore Jespipe Docs »

Report Bug · Request Feature · Discussions · Releases

Jespipe v0 - Experimental

Welcome to the Jespipe repository! Our default branch is for the development of the Jespipe v0 Experimental release. If you are interested in helping us publish a professional, robust release of Jespipe, please check out our contributing guidelines!

Table of Contents

Quick Start

System Requirements

The recommended runtime environment for Jespipe is on High-Performance Computing clusters, but your system only has to meet the following minimum requirements in order to run Jespipe:

Category Minimum Requirement
Operating System: CentOS 7 or newer, Debian 10 or newer, OpenSUSE 15.3 or newer, RedHat Enterprise Linux 7 or newer, Ubuntu 18.04 or newer
Number of Processor Cores: Bare minimum: 2 cores - 4 threads; Recommended: 8 cores - 16 threads
RAM: Bare minimum: 4GB; Recommended: 16GB
GPU: Not required but recommended for larger-scale models and computationally instensive attacks.
GCC Version: GCC version 8.3.1 or greater
Python Version: 3.5+, 3.8+, 3.9.5, or 3.9.6

It is strongly recommended that you use Jespipe on a High-Performance Computing cluster because while Jespipe's system resource consumption is minimal, under-powered machines will be quickly overwhelmed by large-scale adversarial analyses.

Install Dependencies

Use the following commands to install Jespipe's system-level dependencies according to your operating system.

CentOS / RedHat Enterprise Linux:

sudo yum update
sudo yum groupinstall "Development Tools" "Development Libraries"
wget https://www.python.org/ftp/python/3.9.6/Python-3.9.6.tar.xz -O - | tar -xJv
cd Python-3.9.6
./configure --enable-shared --enable-optimizations
make && sudo make install

OpenSUSE:

sudo zypper update
sudo zypper install -t pattern devel_basis
sudo zypper install python39-base python39-devel python39-pip

Ubuntu / Debian:

sudo apt-get update
sudo apt-get install build-essential python3.9-full python3.9-dev python3-pip

Install OpenMPI

Use the following commands to install OpenMPI according to your operating system.

CentOS / RedHat Enterprise Linux:

sudo yum install openmpi openmpi-devel

OpenSUSE:

sudo zypper install openmpi4 openmpi4-libs openmpi4-devel openmpi4-config openmpi4-docs

Ubuntu / Debian:

sudo apt-get install openmpi-bin openmpi-common openmpi-doc

Install Jespipe

With the system-level dependencies and OpenMPI installed on your system, use the following commands to set Jespipe up on your system.

Install from release (for production):

To install Jespipe from a release, first download a release and then use the following commands:

tar -xzvf jespipe-<release-number>.tar.gz
cd jespipe-<release-number>
pip install -r requirements.txt
python setup.py install

Install from source (for development):

To install Jespipe from source, use the following commands:

git clone https://github.com/NucciTheBoss/jespipe.git
cd jespipe
pip install -r requirements.txt
python setup.py install

Test Installation:

If all went well, you should be able to print out Jespipe's version info without issue:

$ python main.py --version

       _                 _                     ___   ___  __ 
      | |               (_)                   / _ \ / _ \/_ |
      | | ___  ___ _ __  _ _ __   ___  __   _| | | | | | || |
  _   | |/ _ \/ __| '_ \| | '_ \ / _ \ \ \ / / | | | | | || |
 | |__| |  __/\__ \ |_) | | |_) |  __/  \ V /| |_| | |_| || |
  \____/ \___||___/ .__/|_| .__/ \___|   \_/  \___(_)___(_)_|
                  | |     | |                                
                  |_|     |_|                                
 

Jespipe-v0.0.1  Copyright (C) 2021  Jason C. Nucciarone, Eric Inae, Sheila Alemany

This program comes with ABSOLUTELY NO WARRANTY; for details please see the LICENSE file. 
This is free software, and you are welcome to redistribute it
under certain conditions; please visit https://github.com/NucciTheBoss/jespipe for more details.

Where to go from here?

  • If this is your first time using Jespipe for conducting adversarial machine learning analysis, please check out our Introduction to Jespipe tutorial!

  • If you are familiar with Jespipe, but like to learn all that Jespipe is capable of, please look through our documentation.

  • If you are an old pro with Jespipe, have fun!

What's Included

jespipe
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── LICENSE_DOCS
├── README.md
├── .config.json
├── main.py
├── requirements.txt
├── setup.py
├── assets
├── contrib
│   ├── build-tools
│   ├── diagrams
│   │   └── drawio
│   └── pseudocode
├── etc
│   ├── skel
│   │   ├── control-templates
│   │   └── plugin-templates
│   │       ├── attack
│   │       ├── manip
│   │       ├── model
│   │       └── plot
│   └── ursula
├── examples
│   ├── control-files
│   ├── datasets
│   │   ├── google-stock
│   │   ├── google-stock-xgb
│   │   └── ozone-level-detection
│   └── plugins
│       ├── attacks
│       ├── manips
│       ├── models
│       │   └── RNN
│       └── plots
├── jespipe
│   └── plugin
│       ├── attack
│       ├── clean
│       ├── manip
│       └── train
└── utils
    ├── appinfo
    ├── filesystem
    ├── managerops
    ├── workeradmin
    └── workerops

There are about 80+ files distributed with a release of Jespipe so it would not be practical to go through all of them, but here is a list of the most important files and directories:

Files

  • README.md - A copy of this README.

  • LICENSE - A copy of the GNU General Public License version 3.

  • .config.json - Default configuration file for Jespipe.

  • main.py - The Jespipe application.

  • requirements.txt - File to install all of Jespipe's external dependencies.

  • setup.py - File to install jespipe-plugin module.

Directories

  • assets - Pictures and text files used by Jespipe.

  • contrib - Files to help with the development of Jespipe.

  • etc - Extra configuration files and skeleton files for Jespipe.

  • examples - Examples for Jespipe.

  • jespipe - The jespipe-plugin module needed for plugins.

  • utils - Helper functions used by Jespipe.

Documentation

Jespipe's documentation is currently hosted using GitHub's Wiki pages here. If you interested in adding to the documentation such as add on to the Wikipages or adding more forms of documentation (manpages, readthedocs, Sphinx, etc.), please check out our contributing guidelines!

Bugs and Feature Requests

Have a bug or feature request that you want to submit? Please read through Jespipe's issue guidelines first and search for existing and closed issues. If your issue or idea has not been addressed yet, feel free to open a new issue.

The Jespipe developers have provided the following issue templates for submitting a bug report and/or feature request:

Contributing

Please read through our contributing guidelines. Included are directions for opening issues, coding standards, and how to submit contributions to Jespipe.

All Python code should follow the standards outlined in PEP 8, PEP 257, and PEP 484. All function and class docstrings should follow the reStructuredText Markup Specification.

Versioning

For transparency into Jespipe's release cycle and in an attempt to support older High-Performance Computing clusters, Jespipe is maintain under the the Semantic Versioning guidelines. Jespipe's developers may sometimes make mistakes, but we will adhere to these rules as best we can.

See the releases section of Jespipe's repository for changelogs for each release version of Jespipe. Sometimes they might not be pretty, but the Jespipe developer's will do their darnest to summarize the most noteworthy changes in each release.

Creators

  • Jason C. Nucciarone - Pennsylvania State University - Lead Author

  • Eric Inae - Andrews University - Attack Plugin Contributor

  • Sheila Alemany - Florida International University - Data Manipulation and Model Training Plugin Contributor

Copyright and License

Code and documentation copyright © 2021 Jason C. Nucciarone, Eric Inae, and Sheila Alemany. Code released under the GNU General Public License version 3. Documentation is released under the BSD 3-Clause License.

Citation

If you use Jespipe for your research, please include the following citation in your paper:

@inproceedings{
  title={Jespipe: A Plugin-Based, Open MPI Framework for Adversarial Machine Learning Analysis},
  author{Alemany, Sheila and Nucciarone, Jason and Pissinou, Niki},
  booktitle={2021 IEEE International Conference on Big Data (Big Data)},
  year={2021},
  organization={IEEE}
}