Releases: OpenCTI-Platform/opencti
Version 3.0.1
OpenCTI version 3.0.1 has been released! We hotfixed 6 bugs linked to the implementation of the RBAC capabilities. On the next milestone, we will work on the test coverage of the platform to strengthen our ability to develop the product and its features in depth. Thank you to all community members who reported these bugs.
Bug Fixes:
- #488 First seen date can be more recent than last seen date
- #511 Unable to add permissions to a group
- #510 Problem when trying to add an external reference to a Threat Report
- #509 Author field does not update after creation of the entity
- #514 Incident view load indefinitely
- #516 User roles/capabilities change doesn't clear the token cache
Version 3.0.0
Dear OpenCTI community, we are proud to announce the release of OpenCTI version 3.0.0. This version is a turning point in the OpenCTI roadmap, as we worked hard on the following objective: allow you to deploy OpenCTI in production within your organization with the right level of security and the expected stability and performances, turn OpenCTI into an enterprise-grade product.
One of the most important enhancement in this version is the implementation of the RBAC system, associated to 3 new authentication strategies (LDAP/AD, OpenID, etc.). You are now able to create roles and assign roles to users to grant them capabilities (read knowledge, update knowledge, import, manage accesses, etc.). The other feature is about reports and data. The import/export system is now stable (with the observed-data management) and we improved again the performances of write operations. We also speeded-up a lot a views (report knowledge graph, listing of some relationships, etc.).
A lot of improvements not related to the main features of this release have been made. We published a new connector to analyze and extract IOCs from PDF files thanks to a member of our community. Many bugfixes on the API and the frontend, new content in the documentation and the creation of the virtual machine template hosting the whole stack for testing purposes. As we prepare a lot of new enrichment connectors for observables, we introduced the max TLP option to avoid leaking sensitive information. Other knowledge connectors will be soon published: AlienVault, CrowdStrike, TheHive, and other vendors.
In a few days, we will send you a message with the date of our first webinar, during which we will present how the platform could be used in different types of organization. As we often say, it is just the beginning of an exciting adventure, with soon much more community activities, data management, intelligent subsystems, visualizations and investigations capabilities.
Enhancements:
- #487 Introduce kill chain view and diamond model
- #484 Automatic completion of marking when creating links
- #467 Enrichment connectors must have a "MAX TLP" config to avoid enrichment on sensitive data
- #466 Report views enhancement
- #442 Huge documentation enhancement
- #403 Creating "targets" relation between an Attack Pattern and a Vulnerability
- #398 Pre-installed OpenCTI iso or ready VM?
- #380 Add inference when action is linked to a specific malware
- #373 LDAP / SSO authentication
- #372 Automatically populate reports
- #330 threats to entities relations
- #329 Organization to threat actor relations
- #328 Organization to organization relations
- #260 Automatically compute the marking of entities/relations from reports
- #182 Change the behavior of auto-complete field
- #148 Workspaces : Add type of entity when selecting an entity for widget creation
- #86 Implement Lockheed Martin Cyber Kill Chain in model
- #75 Implement the RBAC system
- #62 Organisation : reliability
Bug Fixes:
Version 2.1.4
OpenCTI 2.1.4 has been released! This version hotfixes 4 bugs and introduces some technical enhancements in the migration system (no more errors when launching a fresh platform). We also fixed 2 bugs in the Python library that impacted the import of vulnerabilities through the CVE connector. Thank you to all community members who reported these bugs. We will now focus on the next milestone: workspaces, outputs and the full support of LDAP/SSO with roles and permissions (read only, read write, etc.).
Enhancements:
- #445 Fresh platform should not apply migrations
- #413 [Doc] nodejs version on Ubuntu 18.04 is too old
Bug Fixes:
Version 2.1.3
Dear community, OpenCTI version 2.1.3 has been released! We mainly focused our work on the stability of the whole product as well as the resolution of some major bugs. However, several new features have also been implemented, in particular the possibility of exporting lists of entities in STIX2 or CSV format: exporting campaign indicators, reports about an intrusion sets, all malwares, etc. We have completed the refactoring of the Python library and added all the methods that were still missing on some entity types as well as the file upload feature. For the next release, we will carry on our huge refactor of workspaces and statistics in general. We also plan to conduct a documentation working session to improve it in depth.
Last but not least, we will soon plan the organization of a webinar that will be didicated to OpenCTI basics but also a very interesting use case for both red teams and blue teams: how to put all knowledge about an incident in OpenCTI and replay it using Caldera. Double advantage of such a scenario: the capitalization of knowledge related to the incident as well as the ability to test the reaction of SOC / CSIRT in the event of a similar attack. Your feedback and your impressions are precious, do not hesitate to send us your use cases and the difficulties you encounter!
⚠️ Breaking changes ⚠️
Grakn Core Server has been upgraded from 1.5.9 to version 1.6.2. The migration process of existing data cannot be done automatically even if you are using the Grakn Docker container. You have to follow the migration procedure Upgrading an Existing Installation to Grakn Core 1.6.x. available in the Grakn documentation. If you have any trouble to make this work, we are available to help you on our Slack channel or you can reach the Grakn team directly on their channel.
Enhancements:
- #424 Enhance searching of entities
- #412 Add OR/AND option to filter Observables & Indicators
- #396 Infinite loading lists in observables and entities of a report
- #391 Export lists of objects (intrusion sets, indicators, etc.)
- #390 Duplicate function askEnrich in the API
- #388 Display the number of entities in each view/lists
- #371 Migrate to Grakn 1.6.1
- #360 Redirect to the requested page after login
- #345 Add tagging for Tools/Vulnerabilities/Observables/Reports
- #334 Tags for reports
- #123 Implement CSV export
Bug Fixes:
- #432 Uploading a PDF threat report results in "Cannot read property 'toLowerCase' of null" error
- #427 Organisation type vs category
- #419 Filter with no tag not working anymore
- #416 infinite loading of reports using specific sort options
- #415 Mutation intrusionSetAdd creates an intrusion set with the first_seen value as the last_seen value
- #410 Exception when importing STIX 2 Indicators due to invalid default type in valid_from
- #407 Mandatory properties are not filled for inferred relations
- #405 Displaying an indicator constantly refreshes the page
- #404 Cannot create an Indicator of type Mac-Addr or Directory
- #381 New reports are added without a set "Processing Status"?
Version 2.1.2
Dear community, the OpenCTI platform version 2.1.2 has been released! This version provides users of the platform with many new features that will allow them to better modelize their CTI knowledge. We have resolved the gap that existed between the OpenCTI data model and the STIX2 schema by definitively separating the concepts of observables and indicators. It is now possible to create indicator in various formats: STIX Pattern, Snort, Sigma, YARA, etc.
All graphics and visuals are now based on ElasticSearch queries to increase performance. This will allow us to completely rework the workspaces and make them real monitoring tools in the next release. Also we have disabled all inferences rules on the platform by default, so a very important screen has been added to the settings section, allowing users to enable inferences rules if they really need it. The MISP connector has been fully refactored and a documentation is available. Finally, we have added many examples to the Python library.
Enhancements:
- #383 Improve performance by using Elastic when searching for relations (when we can)
- #375 Enhance display of relations and separate inferred/not inferred
- #366 Use ElasticSearch for all statistics (timeseries, distribution, etc.)
- #349 Enhance loaders/spinners everywhere
- #335 Detection rules (yara, snort, suricata, sigma, etc)
- #316 Split indicator/observable concepts, create indicators from observables
- #162 Refactor the observables schema to match STIX2 references
- #145 Enable/disable inferences rules in settings
- #58 Observables : expiration date
- #57 Observables : scoring/rating
Bug Fixes:
- #369 Store the remote IDs in some ES entities is useless and performance killer
Version 2.1.1
OpenCTI 2.1.1 has been released! This version is hotfixing 5 bugs (4 in the API/Frontend and 1 in the Python library) found after the last release. Thank you to all people who reported these bugs so we can now work on the next milestone. The next milestone will be focused on: improving performances of charts and relations display in the UI, development of many outputs and graphics (killchains, diamond model, PDF export of knowledge, full refactor/enhancement of workspaces, graph view of entities, comparison of threats TTPs/infrastructure and introduction of indicator concept.
Bug Fixes:
Version 2.1.0
Dear community, the OpenCTI platform version 2.1.0 has been released! This version is an important step for the future developments of OpenCTI as a full Cyber Threat Intelligence product. We have worked on major issues and features directly linked to what you can expect from OpenCTI and what we need, as a developers team, to build a powerful and durable application. We have done a lot of work on indexing in ElasticSearch and in general all the way API methods are organized (removing more than 7K lines of useless source code). Ingestion and reading performance have been improved by 12x or by 20x in some cases.
We have also completed the data model and have introduced very useful features to allow you to fully modelize threats that may target your organization. You are now able to directly link an observable on a relation "threat/incident => uses => TTP", for instance to indicate the registry key used for persistence or the sender email address of the phishing message. The attack patterns list has been reshaped to a true killchain with the description of each relation to ensure a better understanding of analysts. Observables can now be linked together, allowing you for instance to link hashes together if it corresponds to the same file, or link an IP address that resolves a domain name.
Last but not least, we have fully refactored the Python library and started to write a proper documentation, you have now access to many useful examples to interact with the OpenCTI platform in the Github repository. We will continue our efforts to make OpenCTI an indispensable tool for CTI, SOC and CSIRT teams around the world. We will soon publish usage and integration tips in existing workflows and plan a usecases-oriented webinar in January 2020.
Enhancements:
- #351 Be able to reset the state of a connector in the UI
- #339 ATT&CK techniques not searchable with their code
- #336 Add the ID of Attack Patterns
- #332 Observables must be able to indicate relations
- #319 Technical error thrown when not logged in
- #317 Observables filtering
- #315 Add Minio version in the "About tab"
- #314 Global performances improvement
- #308 Relations between observables
- #268 Global search in parameter of URL
- #266 Add a tags field on creation forms
- #245 Killchain view for Attack Patterns
- #219 Unable to add "localized in" relation
- #109 Import is really really slow
- #67 Export all entities to STIX2 JSON
Bug Fixes:
Version 2.0.2
OpenCTI 2.0.2 has been released! This version is mainly focused on fixing bugs, one affecting the graph database that could trigger out of memory issues in Grakn and the other leading to lost data in the workers if the API is not available. Please stay tuned for the next milestone (2.1.0) which will includes a huge work about data ingestion performances. We already know that this is the one of the most important weakness of the platform right now! Do not hesitate to send us your feedbacks on our last releases!
Enhancements:
- #309 Prevent worker for consuming messages if the API is down
- #298 Missing script for 'npm run schema'
Bug Fixes:
Version 2.0.1
We just released OpenCTI version 2.0.1! After the 2.0.0 some users reported us important bugs that are now fixed in this new version (especially on connectors & worker). We also introduced a persistent states in external import connectors such as MITRE or CVE to avoid re-sending messages to the queue each time the connector is restarted. Our work in the next milestones will be focused on improving the data ingestion speed and developing features to help users to massively handle entities and relationships in the platform (detect duplicates, merge, split, bulk delete, etc.).
Enhancements:
- #291 Add simple state management for connectors.
- #237 Improve search engine capacity
- #196 OpenCTI development environment documentation is outdated
Bug Fixes:
- #287 Subsectors cannot be added to sectors
- #285 Worker stopped consuming messages some processing
- #277 no inference relationship
- #276 Cannot export some reports
- #216 Searching for Entities returns inconsistent results
- #211 Multiple workers and PermanentBackendException: Permanent failure in storage backend
Version 2.0.0
We are proud to announce a new major release of the OpenCTI platform: 2.0.0 is out! Although the documentation is still under construction, this new version brings many features and improvements to users. It allows you to store and manage files, add tags to entities, easily create relationships to relationships in reports, and, depending on the available connectors, enable automatic enrichment on observables, extraction of indicators in PDF files and exports in different formats. Several bugs have been fixed and multiple improvements made in display and performance. We are waiting for your feedback and future contributions, especially on connectors!
⚠️ Breaking changes ⚠️
New dependency
- To handle file storage for import, export and files linked to entities, Minio has been introduced in the OpenCTI stack as a required component. In the future, any S3 storage system will be able to store the OpenCTI data and files.
- The file management system can be used by connectors to extract intelligence such as IoCs, TTPs or store any export from the platform (generated PDFs, STIX2, etc.).
Workers and connectors
- There is now only one worker for writing data coming from the RabbitMQ broker on the platform, so the
exportworker is deprecated. The worker remain the same base code, the parametertypeis no longer required. - To handle import and export (only STIX2 for the moment), 2 new connectors have been introduced.
- For the worker and connectors configuration, the RabbitMQ parameters are no longer needed, only the OpenCTI API hostname and token are required. RabbitMQ parameters are provided by the API through the Python helpers.
The new configuration of connectors is available in the dedicated documentation.
Enhancements:
- #254 Separate observables list of reports in a different QueryRenderer
- #249 Create new attack pattern to be associated to a report
- #244 Add a "drops" relation between malwares/tools.
- #241 Enhance the custom attributes management and update
- #236 Add version/build number and minimal system info in dashboard
- #232 Aliases display enhancement
- #229 Global tagging system
- #221 5 level certainty scale not adaptable
- #217 Better handling of concurrent integration
- #212 Remove "waiting behavior" from entrypoint, let docker restart the containers
- #204 Redesign the connector status page
- #191 Reduce opencti/platform docker image size
- #170 Add standalone observables
- #141 Observables don't appear when importing a file
- #130 Introduce file storage for export download
- #105 Add Kill Chain Phase selection when adding observable
- #69 Enhance knowledge graph of reports
- #61 Organisation : associated IP addresses, domain names, URL-s
- #48 Implement the observable enrichment
- #44 Attach files to report
- #43 Differenciate the display of sectors that are subsectors
- #42 Add relationships and knowledge everywhere
- #39 Add aliases to the generic entity creation form
- #38 Automatic graph organization on report
- #37 Display marking definitions in all entities / relations
- #34 Display entity information in a graph view
Bug Fixes:
- #235 The entity "Region" can't be added as the location property of a relation.
- #228 Inferred relations not displayed in the relationships lists
- #220 Inferred relation instrusion set - country - region
- #210 Unable to create a "Workspace" in the "Explore" view
- #209 Observables of entities cannot be sorted
- #136 Marking color