Perform data science on data that remains in someone else's server

Quickstart

✅ Linux ✅ macOS ✅ Windows ✅ Docker ✅ Kubernetes

Install Client

$ pip install -U syft[data_science]

Launch Server

# from Jupyter / Python
import syft as sy
sy.requires(">=0.8.6,<0.8.7")
node = sy.orchestra.launch(
    name="my-domain",
    port=8080,
    create_producer=True,
    n_consumers=1,
    dev_mode=True,
    reset=True, # resets database
)

# or from the command line
$ syft launch --name=my-domain --port=8080 --reset=True

Starting syft-node server on 0.0.0.0:8080

Launch Client

import syft as sy
sy.requires(">=0.8.6,<0.8.7")
domain_client = sy.login(
    port=8080,
    email="info@openmined.org",
    password="changethis"
)

PySyft in 10 minutes

📝 API Example Notebooks

• 00-load-data.ipynb
• 01-submit-code.ipynb
• 02-review-code-and-approve.ipynb
• 03-data-scientist-download-result.ipynb
• 04-pytorch-example.ipynb
• 05-custom-policy.ipynb
• 06-multiple-code-requests.ipynb
• 07-domain-register-control-flow.ipynb
• 08-code-version.ipynb
• 09-blob-storage.ipynb
• 10-container-images.ipynb
• 11-container-images-k8s.ipynb
• 12-custom-api-endpoint.ipynb

Deploy Kubernetes Helm Chart

0. Deploy Kubernetes

Required resources: 1 CPU and 4GB RAM. However, you will need some special instructions to deploy, please consult these instructions or look at the resource constraint testing here.
Recommended resources: 8+ Cores and 16GB RAM

If you're using Docker Desktop to deploy your Kubernetes, you may need to go into Settings > Resources and increase CPUs and Memory.

Note: Assuming we have a Kubernetes cluster already setup.

1. Add and update Helm repo for Syft

helm repo add openmined https://openmined.github.io/PySyft/helm
helm repo update openmined

2. Search for available Syft versions

helm search repo openmined/syft --versions --devel

3. Set your preferred Syft Chart version

SYFT_VERSION="<paste the chart version number>"

4. Provisioning Helm Charts

helm install my-domain openmined/syft --version $SYFT_VERSION --namespace syft --create-namespace --set ingress.className="traefik"

Ingress Controllers

For Azure AKS

helm install ... --set ingress.className="azure-application-gateway"

For AWS EKS

helm install ... --set ingress.className="alb"

For Google GKE we need the gce annotation.

helm install ... --set ingress.class="gce"

Note:

🚨 Our old deployment tool HAGrid has been deprecated. For the updated deployment options kindly refer to:

• 📚 Deployments

Docs and Support

• 📚 Docs
• #support on Slack

Install Notes

• PySyft 0.8.6 Requires: 🐍 python 3.10 - 3.12 - Run: pip install -U syft
• PyGrid Requires: 🐳 docker or ☸️ kubernetes

Versions

0.9.0 - Coming soon...
0.8.7 (Beta) - dev branch 👈🏽 API - Coming soon...
0.8.6 (Stable) - API

Deprecated:

• 0.8.5-post.2 - API
• 0.8.4 - API
• 0.8.3 - API
• 0.8.2 - API
• 0.8.1 - API
• 0.8.0 - API
• 0.7.0 - Course 3 Updated
• 0.6.0 - Course 3
• 0.5.1 - Course 2 + M1 Hotfix
• 0.2.0 - 0.5.0

PySyft and PyGrid use the same version and its best to match them up where possible. We release weekly betas which can be used in each context:

PySyft (Stable): pip install -U syft

PySyft (Beta): pip install -U syft --pre

What is Syft?

Syft is OpenMined's open source stack that provides secure and private Data Science in Python. Syft decouples private data from model training, using techniques like Federated Learning, Differential Privacy, and Encrypted Computation. This is done with a numpy-like interface and integration with Deep Learning frameworks, so that you as a Data Scientist can maintain your current workflow while using these new privacy-enhancing techniques.

Why should I use Syft?

Syft allows a Data Scientist to ask questions about a dataset and, within privacy limits set by the data owner, get answers to those questions, all without obtaining a copy of the data itself. We call this process Remote Data Science. It means in a wide variety of domains across society, the current risks of sharing information (copying data) with someone such as, privacy invasion, IP theft and blackmail will no longer prevent the vast benefits such as innovation, insights and scientific discovery which secure access will provide.

No more cold calls to get access to a dataset. No more weeks of wait times to get a result on your query. It also means 1000x more data in every domain. PySyft opens the doors to a streamlined Data Scientist workflow, all with the individual's privacy at its heart.

Terminology

👨🏻‍💼 Data Owners	👩🏽‍🔬 Data Scientists
Provide datasets which they would like to make available for study by an outside party they may or may not fully trust has good intentions.	Are end users who desire to perform computations or answer a specific question using one or more data owners' datasets.
🏰 Domain Server	🔗 Gateway Server
Manages the remote study of the data by a Data Scientist and allows the Data Owner to manage the data and control the privacy guarantees of the subjects under study. It also acts as a gatekeeper for the Data Scientist's access to the data to compute and experiment with the results.	Provides services to a group of Data Owners and Data Scientists, such as dataset search and bulk project approval (legal / technical) to participate in a project. A gateway server acts as a bridge between it's members (Domains) and their subscribers (Data Scientists) and can provide access to a collection of domains at once.

Community

	🎥 PETs: Remote Data Science Unleashed - R gov 2021 🎥 Introduction to Remote Data Science - PyTorch 2021 🎥 The Future of AI Tools - PyTorch 2020 🎥 Privacy Preserving AI - MIT Deep Learning Series 🎥 Privacy-Preserving Data Science - TWiML Talk #241 🎥 Privacy Preserving AI - PyTorch Devcon 2019 📖 Towards general-purpose infrastructure for protect... 📖 Syft 0.5: A platform for universally deployable ... 📖 A generic framework for privacy preserving deep ...

Courses

Contributors

OpenMined and Syft appreciates all contributors, if you would like to fix a bug or suggest a new feature, please see our guidelines.

Supporters

Disclaimer

Syft is under active development and is not yet ready for pilots on private data without our assistance. As early access participants, please contact us via Slack or email if you would like to ask a question or have a use case that you would like to discuss.

License

Apache License 2.0
Person icons created by Freepik - Flaticon