[documentation] Release the new version

opencti-docker/docker-compose.yml

@@ -29,9 +29,7 @@ services:
       - RABBITMQ_DEFAULT_PASS=guest    
     restart: always
   worker-import:
-    image: opencti/worker:rolling
-    build:
-      context: ../opencti-worker
+    image: opencti/worker:1.1.0
     environment:
       - WORKER_TYPE=import
       - WORKER_LOG_LEVEL=info
@@ -50,9 +48,7 @@ services:
       replicas: 4
     restart: always
   worker-export:
-    image: opencti/worker:rolling
-    build:
-      context: ../opencti-worker
+    image: opencti/worker:1.1.0
     environment:
       - WORKER_TYPE=export
       - WORKER_LOG_LEVEL=info
@@ -71,13 +67,11 @@ services:
       replicas: 2      
     restart: always    
   opencti:
-    image: opencti/platform:rolling
-    build:
-      context: ../opencti-platform    
+    image: opencti/platform:1.1.0
     environment:
       - APP__PORT=8080
       - APP__ADMIN__EMAIL=admin@opencti.io
-      - APP__ADMIN__PASSWORD=admin
+      - APP__ADMIN__PASSWORD=ChangeMe
       - APP__ADMIN__TOKEN=ChangeMe
       - APP__LOGS=./logs
       - APP__REACTIVE=true
@@ -104,7 +98,7 @@ services:
       - rabbitmq
     restart: always
   connector-opencti:
-    image: opencti/connector-opencti:rolling
+    image: opencti/connector-opencti:1.1.0
     environment:
       - RABBITMQ_HOSTNAME=rabbitmq
       - RABBITMQ_PORT=5672 

opencti-documentation/docs/development/connectors.md

@@ -4,34 +4,98 @@ title: Development of connectors
 sidebar_label: Connectors
 ---
 
-> Available from the version 1.1.0
-
 Connectors are the cornerstone of the OpenCTI platform and allow organizations to easily ingest new data on the platform. The OpenCTI core development team will provide as many connectors as they can but any developers can contribute to community connectors provided on the [official repository](https://github.com/OpenCTI-Platform/connectors).
 
 ## Introduction
 
-We choose to have a very decentralized approach on connectors, in order to bring a maximum freedom to developers and vendors. So a connector on OpenCTI can be defined by **a standalone Python 3 process that pushes an understandable format of data to an ingestion queue of messages**. For the moment, only a valid STIX2 bundle is supported, by we intend to support CSV and other formats in the future.
+We choose to have a very decentralized approach on connectors, in order to bring a maximum freedom to developers and vendors. So a connector on OpenCTI can be defined by **a standalone Python 3 process that pushes an understandable format of data to an ingestion queue of messages**.
 
-![Connector architecture](assets/development/connector_architecture.png "Connector architecture")
+> For the moment, only a valid STIX2 bundle is supported, by we intend to support CSV and other formats in the future.
 
-## Development
+![Connector architecture](assets/development/connector_architecture.png "Connector architecture")
 
 Each connector must implement a long-running process that can be launched just by executing the main Python file. The only mandatory dependency is the `OpenCTIConnectorHelper` class that enables the connector to send data to OpenCTI.
 
-### Connector configuration
+## Connector configuration
 
-The connector configuration can be based on a `config.yml` located in the same directory than the main file or in environments variables when using Docker. The only 2 mandatory fields are `name` and `confidence_level`. 
+The connector configuration can be based on a `config.yml` located in the same directory than the main file or in environments variables when using Docker.
 
+> In the configuration, the RabbitMQ configuration is mandatory, as well as the `name` and the `confidence_level` (that will be used to solve conflicts between entities or relationships).
 
+Here is an example of a simple `config.yml` file:
+
+```yaml
+rabbitmq:
+  hostname: 'localhost'
+  port: 5672
+  username: 'guest'
+  password: 'guest'
+
+connector:
+  name: 'Connector instance'
+  confidence_level: 3
+  log_level: 'info'
 ```
+
+> For environement variables, your connector must respect the standard mapping of configuration, replacing each list level by the char `_`. For instance, the configuration `config['connector']['server']['hostname']` can be set as an environement variable named `CONNECTOR_SERVER_HOSTNAME`.
+
+```python
+class Connector:
+    def __init__(self):
+        # Get configuration
+        config_file_path = os.path.dirname(os.path.abspath(__file__)) + '/config.yml'
+        self.config = dict()
+        if os.path.isfile(config_file_path):
+            config = yaml.load(open(config_file_path), Loader=yaml.FullLoader)
+            self.config_rabbitmq = config['rabbitmq']
+            self.config['name'] = config['connector']['name']
+            self.config['confidence_level'] = config['connector']['confidence_level']
+            self.config['log_level'] = config['connector']['log_level']
+        else:
+            self.config_rabbitmq = dict()
+            self.config_rabbitmq['hostname'] = os.getenv('RABBITMQ_HOSTNAME', 'localhost')
+            self.config_rabbitmq['port'] = os.getenv('RABBITMQ_PORT', 5672)
+            self.config_rabbitmq['username'] = os.getenv('RABBITMQ_USERNAME', 'guest')
+            self.config_rabbitmq['password'] = os.getenv('RABBITMQ_PASSWORD', 'guest')
+            self.config['name'] = os.getenv('CONNECTOR_NAME', 'Connector instance')
+            self.config['confidence_level'] = int(os.getenv('CONNECTOR_CONFIDENCE_LEVEL', 3))
+            self.config['log_level'] = os.getenv('CONNECTOR_LOG_LEVEL', 'info')
+```
+
+## Initialize the OpenCTI connector helper
+
+After getting the configuration parameters of your connector, you have to initialize the OpenCTI connector helper by using the `pycti` Python library.
+
+```python
 from pycti import OpenCTIConnectorHelper
 
-connector_identifier = instance_name # where instance_name is lowercase and contains no special chars, unique based 
+connector_identifier = instance_name # where instance_name is lowercase and contains no special chars, unique based
+# connector_identifier = ''.join(e for e in self.config['name'] if e.isalnum()).lower()
 
-opencti_connector_helper = OpenCTIConnectorHelper(
+self.opencti_connector_helper = OpenCTIConnectorHelper(
 	connector_identifier,
-    config_connector,
-    config_rabbitmq,
+    self.config, # the configuration of the connector
+    self.config_rabbitmq, # the RabbitMQ configuration with hostname, port, username and password
     'info' # info, warning, error
 )
-```
+```
+
+## Send data to OpenCTI
+
+The OpenCTI connector helper method `send_stix2_bundle` must be used to send data to OpenCTI. Other methods such as `send_csv` will be implemented in the future. The `send_stix2_bundle` function takes 2 arguments.
+
+1. A serialized STIX2 bundle as a `string` (mandatory)
+2. A `list` of entities types that should be ingested (optional)
+
+Here is an example using the STIX2 Python library:
+
+```python
+from stix2 import Bundle
+
+bundle = Bundle(objects=bundle_objects).serialize()
+self.opencti_connector_helper.send_stix2_bundle(bundle)
+```
+
+## Examples
+
+You can read the source code of the OpenCTI connectors directly in the [dedicated repository](https://github.com/OpenCTI-Platform/connectors).

opencti-documentation/docs/installation/connectors.md

@@ -87,5 +87,8 @@ Change the `config.yml` content according to the parameters of the platform and
 $ python3 misp.py
 ```
 
+## Connectors status
 
+The connector status can be displayed in the dedicated section. You will be able to see the statistics of the RabbitMQ queue of the connector:
 
+![Connectors status](assets/installation/connectors_status.png "Connectors status")

opencti-documentation/website/package.json

@@ -10,5 +10,6 @@
   },
   "devDependencies": {
     "docusaurus": "^1.9.0"
-  }
+  },
+  "version": "1.1.0"
 }

opencti-documentation/website/pages/en/versions.js

@@ -83,12 +83,12 @@ function Versions(props) {
                         {/* You are supposed to change this href where appropriate
                         Example: href="<baseUrl>/docs(/:language)/:version/:id" */}
                         <a
-                          href={`${siteConfig.baseUrl}${siteConfig.docsUrl}getting-started/introduction`}>
+                          href={`${siteConfig.baseUrl}${siteConfig.docsUrl}${version}/getting-started/introduction`}>
                           Documentation
                         </a>
                       </td>
                       <td>
-                        <a href={`${repoUrl}/releases/tag/v${version}`}>
+                        <a href={`${repoUrl}/releases/tag/${version}`}>
                           Release Notes
                         </a>
                       </td>

opencti-documentation/website/versioned_docs/version-1.1.0/development/connectors.md

@@ -0,0 +1,102 @@
+---
+id: version-1.1.0-connectors
+title: Development of connectors
+sidebar_label: Connectors
+original_id: connectors
+---
+
+Connectors are the cornerstone of the OpenCTI platform and allow organizations to easily ingest new data on the platform. The OpenCTI core development team will provide as many connectors as they can but any developers can contribute to community connectors provided on the [official repository](https://github.com/OpenCTI-Platform/connectors).
+
+## Introduction
+
+We choose to have a very decentralized approach on connectors, in order to bring a maximum freedom to developers and vendors. So a connector on OpenCTI can be defined by **a standalone Python 3 process that pushes an understandable format of data to an ingestion queue of messages**.
+
+> For the moment, only a valid STIX2 bundle is supported, by we intend to support CSV and other formats in the future.
+
+![Connector architecture](assets/development/connector_architecture.png "Connector architecture")
+
+Each connector must implement a long-running process that can be launched just by executing the main Python file. The only mandatory dependency is the `OpenCTIConnectorHelper` class that enables the connector to send data to OpenCTI.
+
+## Connector configuration
+
+The connector configuration can be based on a `config.yml` located in the same directory than the main file or in environments variables when using Docker.
+
+> In the configuration, the RabbitMQ configuration is mandatory, as well as the `name` and the `confidence_level` (that will be used to solve conflicts between entities or relationships).
+
+Here is an example of a simple `config.yml` file:
+
+```yaml
+rabbitmq:
+  hostname: 'localhost'
+  port: 5672
+  username: 'guest'
+  password: 'guest'
+
+connector:
+  name: 'Connector instance'
+  confidence_level: 3
+  log_level: 'info'
+```
+
+> For environement variables, your connector must respect the standard mapping of configuration, replacing each list level by the char `_`. For instance, the configuration `config['connector']['server']['hostname']` can be set as an environement variable named `CONNECTOR_SERVER_HOSTNAME`.
+
+```python
+class Connector:
+    def __init__(self):
+        # Get configuration
+        config_file_path = os.path.dirname(os.path.abspath(__file__)) + '/config.yml'
+        self.config = dict()
+        if os.path.isfile(config_file_path):
+            config = yaml.load(open(config_file_path), Loader=yaml.FullLoader)
+            self.config_rabbitmq = config['rabbitmq']
+            self.config['name'] = config['connector']['name']
+            self.config['confidence_level'] = config['connector']['confidence_level']
+            self.config['log_level'] = config['connector']['log_level']
+        else:
+            self.config_rabbitmq = dict()
+            self.config_rabbitmq['hostname'] = os.getenv('RABBITMQ_HOSTNAME', 'localhost')
+            self.config_rabbitmq['port'] = os.getenv('RABBITMQ_PORT', 5672)
+            self.config_rabbitmq['username'] = os.getenv('RABBITMQ_USERNAME', 'guest')
+            self.config_rabbitmq['password'] = os.getenv('RABBITMQ_PASSWORD', 'guest')
+            self.config['name'] = os.getenv('CONNECTOR_NAME', 'Connector instance')
+            self.config['confidence_level'] = int(os.getenv('CONNECTOR_CONFIDENCE_LEVEL', 3))
+            self.config['log_level'] = os.getenv('CONNECTOR_LOG_LEVEL', 'info')
+```
+
+## Initialize the OpenCTI connector helper
+
+After getting the configuration parameters of your connector, you have to initialize the OpenCTI connector helper by using the `pycti` Python library.
+
+```python
+from pycti import OpenCTIConnectorHelper
+
+connector_identifier = instance_name # where instance_name is lowercase and contains no special chars, unique based
+# connector_identifier = ''.join(e for e in self.config['name'] if e.isalnum()).lower()
+
+self.opencti_connector_helper = OpenCTIConnectorHelper(
+	connector_identifier,
+    self.config, # the configuration of the connector
+    self.config_rabbitmq, # the RabbitMQ configuration with hostname, port, username and password
+    'info' # info, warning, error
+)
+```
+
+## Send data to OpenCTI
+
+The OpenCTI connector helper method `send_stix2_bundle` must be used to send data to OpenCTI. Other methods such as `send_csv` will be implemented in the future. The `send_stix2_bundle` function takes 2 arguments.
+
+1. A serialized STIX2 bundle as a `string` (mandatory)
+2. A `list` of entities types that should be ingested (optional)
+
+Here is an example using the STIX2 Python library:
+
+```python
+from stix2 import Bundle
+
+bundle = Bundle(objects=bundle_objects).serialize()
+self.opencti_connector_helper.send_stix2_bundle(bundle)
+```
+
+## Examples
+
+You can read the source code of the OpenCTI connectors directly in the [dedicated repository](https://github.com/OpenCTI-Platform/connectors).

opencti-documentation/website/versioned_docs/version-1.1.0/installation/connectors.md

@@ -0,0 +1,95 @@
+---
+id: version-1.1.0-connectors
+title: Connectors activation
+sidebar_label: Enable connectors
+original_id: connectors
+---
+
+## Introduction
+
+Connectors are standalone processes that are independant of the rest of the platform. They are using RabbitMQ to push data to OpenCTI, through a dedicated queue for each instance of connector. Depending on your deployment, you can enable connectors by using the connectors Docker images or launch them manually. 
+
+## Connector configurations
+
+All connectors have 2 mandatory configuration parameters, the `name` and the `confidence_level`. The `name` is the name of the instance of the connector. For instance, for the MISP connector, you can launch as many MISP connectors as you need, if you need to pull data from multiple MISP instances. 
+
+> The `name` of each instance of connector must be unique.
+
+> The `confidence_level` of the connector will be used to set the `confidence_level` of the relationships created by the connector. If a connector needs to create a relationship that already exists, it will check the current `confidence_level` and if it is lower than its own, it will update the relationship with the new information. If it is higher, it will do nothing and keep the existing relationship.
+
+## Docker activation
+
+You can either directly run the Docker image of connectors or add them to your current `docker-compose.yml` file.
+
+### Add a connector to your deployement
+
+For instance, to enable the MISP connector, you can add a new service to your `docker-compose.yml` file:
+
+```
+  connector-misp:
+    image: opencti/connector-misp:1.1.0
+    environment:
+      - RABBITMQ_HOSTNAME=localhost
+      - RABBITMQ_PORT=5672 
+      - RABBITMQ_USERNAME=guest
+      - RABBITMQ_PASSWORD=guest
+      - MISP_NAME=MISP\ Circle
+      - MISP_CONFIDENCE_LEVEL=3
+      - MISP_URL=http://localhost
+      - MISP_KEY=ChangeMe
+      - MISP_TAG=OpenCTI:\ Import
+      - MISP_UNTAG_EVENT=true
+      - MISP_IMPORTED_TAG=OpenCTI:\ Imported
+      - MISP_INTERVAL=1 # Minutes
+      - MISP_LOG_LEVEL=info
+    restart: always
+ ```
+
+### Launch a standalone connector
+
+To launch standalone connector, you can use the `docker-compose.yml` file of the connector itself. Just pull the [dedicated repository](https://github.com/OpenCTI-Platform/connectors) and start the connector:
+
+```
+$ git clone https://github.com/OpenCTI-Platform/connectors
+$ cd misp
+```
+
+Change the configuration in the `docker-compose.yml` accoarding to the parameters of the platform and of the targeted service. RabbitMQ credentials are the only parameters that the connector need to send data to OpenCTI. Then launch the connector:
+
+```
+$ docker-compose up
+```
+
+## Manual activation
+
+If you want to manually launch connector, you just have to install Python 3 and pip3 for dependencies:
+
+```
+$ apt install python3 python3-pip
+```
+
+Clone the [repository](https://github.com/OpenCTI-Platform/connectors) of the connectors:
+
+```
+$ git clone https://github.com/OpenCTI-Platform/connectors
+$ cd misp/src
+```
+
+Install dependencies and initialize the configuration:
+
+```
+$ pip3 install -r requirements.txt
+$ cp config.yml.sample config.yml
+```
+
+Change the `config.yml` content according to the parameters of the platform and of the targeted service and launch the connector:
+
+```
+$ python3 misp.py
+```
+
+## Connectors status
+
+The connector status can be displayed in the dedicated section. You will be able to see the statistics of the RabbitMQ queue of the connector:
+
+![Connectors status](assets/installation/connectors_status.png "Connectors status")