repo-sync-2023-12-06T17:18:42+0800 (#5)

README.md

@@ -6,21 +6,141 @@ The repo of Private Set Intersection(PSI) and Private Information Retrieval(PIR)
 
 This repo is formerly psi/pir part from secretflow/spu repo.
 
-## Quick Start
+## PSI Quick Start with v2 API
+
+For PSI v1 API and PIR, please check [documentation](https://www.secretflow.org.cn/docs/psi).
+
+### Release Docker
+
+In the following example, we are going to run PSI at a single host.
+
+1. Check official release docker image at [dockerhub](https://hub.docker.com/r/secretflow/psi-anolis8). We also have mirrors at Alibaba Cloud: secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8.
+
+2. Prepare data and config.
+
+receiver.config:
+
+```json
+{
+    "protocol_config": {
+        "protocol": "PROTOCOL_KKRT",
+        "role": "ROLE_RECEIVER",
+        "broadcast_result": true
+    },
+    "input_config": {
+        "type": "IO_TYPE_FILE_CSV",
+        "path": "/root/receiver/receiver_input.csv"
+    },
+    "output_config": {
+        "type": "IO_TYPE_FILE_CSV",
+        "path": "/root/receiver/receiver_output.csv"
+    },
+    "link_config": {
+        "parties": [
+            {
+                "id": "receiver",
+                "host": "127.0.0.1:5300"
+            },
+            {
+                "id": "sender",
+                "host": "127.0.0.1:5400"
+            }
+        ]
+    },
+    "self_link_party": "receiver",
+    "keys": [
+        "id0",
+        "id1"
+    ],
+    "debug_options": {
+        "trace_path": "/root/receiver/receiver.trace"
+    },
+    "check_duplicates": false,
+    "sort_output": false,
+    "recovery_config": {
+        "enabled": false,
+    }
+}
+```
+
+sender.config:
+
+```json
+{
+    "protocol_config": {
+        "protocol": "PROTOCOL_KKRT",
+        "role": "ROLE_SENDER",
+        "broadcast_result": true
+    },
+    "input_config": {
+        "type": "IO_TYPE_FILE_CSV",
+        "path": "/root/sender/sender_input.csv"
+    },
+    "output_config": {
+        "type": "IO_TYPE_FILE_CSV",
+        "path": "/root/sender/sender_output.csv"
+    },
+    "link_config": {
+        "parties": [
+            {
+                "id": "receiver",
+                "host": "127.0.0.1:5300"
+            },
+            {
+                "id": "sender",
+                "host": "127.0.0.1:5400"
+            }
+        ]
+    },
+    "self_link_party": "sender",
+    "keys": [
+        "id0",
+        "id1"
+    ],
+    "debug_options": {
+        "trace_path": "/root/sender/sender.trace"
+    },
+    "check_duplicates": false,
+    "sort_output": false,
+    "recovery_config": {
+        "enabled": false,
+    }
+}
+```
+
+| File Name          | Location                            | Description                                                                |
+| :----------------  | :---------------------------------- | :------------------------------------------------------------------------- |
+| receiver.config    | /tmp/receiver/receiver.config       | Config for receiver.                                                       |
+| sender.config      | /tmp/sender/sender.config           | Config for sender.                                                         |
+| receiver_input.csv | /tmp/receiver/receiver_input.config | Input for receiver. Make sure the file contains two id keys - id0 and id1. |
+| sender_input.csv   | /tmp/sender/sender_input.config     | Input for sender. Make sure the file contains two id keys - id0 and id1.   |
+
+
+3. Run PSI
+
+In the first terminal, run the following command
 
-<!-- todo -->
+```bash
+docker run -it  --rm  --network host --mount type=bind,source=/tmp/receiver,target=/root/receiver -w /root  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:0.1.0beta bash -c "./main --config receiver/receiver.config"
+```
+
+In the other terminal, run the following command simultaneously.
+
+```bash
+docker run -it  --rm  --network host --mount type=bind,source=/tmp/sender,target=/root/sender -w /root  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --cap-add=NET_ADMIN --privileged=true secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/psi-anolis8:0.1.0beta bash -c "./main --config sender/sender.config"
+```
 
 ## Building SecretFlow PSI Library
 
 ### System Setup
 
 
-#### Docker
+#### Dev Docker
 
 We use the same dev docker from secretflow/ubuntu-base-ci.
 
 ```sh
-## start container
+# start container
 docker run -d -it --name psi-dev-$(whoami) \
          --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
          -w /home/admin/dev \

RELEASE.md

@@ -0,0 +1,14 @@
+> NOTE:
+>
+> - `[Feature]` prefix for new features.
+> - `[Bugfix]` prefix for bug fixes.
+> - `[API]` prefix for API changes.
+
+
+## v0.1.0beta
+
+- [API] Add PSI v2 API.
+- [Feature] Add RR22 protocol.
+- [Feature] Support recovery from failure in v2 API.
+- [Feature] Support inner join in v2 API.
+- [Feature] Migrate ECDH, KKRT, RR22 protocol in v2 API.

docs/.gitignore

@@ -0,0 +1 @@
+_build

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/conf.py

@@ -0,0 +1,82 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = "SecretFlow PSI Library"
+copyright = "2023, SecretFlow authors"
+author = "SecretFlow authors"
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    "myst_parser",
+    "sphinx.ext.extlinks",
+]
+
+templates_path = ["_templates"]
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = "alabaster"
+html_static_path = ["_static"]
+
+# -- Options for  sphinx-int -------------------------------------------------
+locale_dirs = ["locale/"]  # path is example but recommended.
+gettext_compact = False  # optional.
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+# global variables
+extlinks = {
+    "psi_doc_host": ("https://www.secretflow.org.cn/docs/psi/en/", "doc "),
+    "psi_code_host": ("https://github.com/secretflow", "code "),
+}
+
+html_theme = "pydata_sphinx_theme"
+html_static_path = ["_static"]
+
+# Enable TODO
+todo_include_todos = True
+
+autodoc_default_options = {
+    "members": True,
+    "member-order": "bysource",
+    "special-members": "__init__",
+    "undoc-members": False,
+    "show-inheritance": False,
+}
+
+
+html_favicon = "_static/favicon.ico"
+
+html_css_files = [
+    "css/custom.css",
+]
+
+html_js_files = ["js/custom.js"]
+
+html_theme_options = {
+    "icon_links": [
+        {
+            "name": "GitHub",
+            "url": "https://github.com/secretflow/psi",
+            "icon": "fab fa-github-square",
+            "type": "fontawesome",
+        },
+    ],
+    "logo": {
+        "text": "Secretflow PSI Library",
+    },
+    "show_nav_level": 4,
+    "language_switch_button": True,
+}

docs/development/index.rst

@@ -0,0 +1,9 @@
+Development
+===========
+
+Advanced topics related to principles, designs, benchmark and more.
+
+.. toctree::
+   :maxdepth: 1
+
+   psi_protocol_intro