- Stage: 0 (strawperson)
- Date: 2023-07-07
This proposal extends the existing ECS field set to store inventory metadata for hosts and users from external application repositories. Using ECS to store such fields will improve metadata querying and retrieval across various use cases.
Terminologies:
The Entity Analytics
initiative within Security refers to hosts and users as entities
. Other generic security and observability use cases may refer to hosts/ users as assets
. Certain directory services or asset management applications use the term 'device' when referring to a host. In this RFC, I have simplified these terminologies to users
and hosts
and these will represent all the neighboring terms.
This proposal includes the following:
- Additional fields in the
users
andos
objects. - Introduces a new field set called
assets
.
This proposal will also facilitate storing host and user inventory within the security solution (the entity store).
Field | Type | Example | Description |
---|---|---|---|
user.profile.id | keyword | 1234 | User ID from the identity datasource. |
user.profile.type | keyword | Employee | Type of user account. |
user.profile.status | keyword | On board | Status of the user account. |
user.profile.first_name | keyword | First | First Name of the User. |
user.profile.last_name | keyword | Last | Last Name of the user. |
user.profile.other_identities | keyword, text | first.last@elk.elastic.co | Array of additional user identities (usually email addresses). |
user.profile.manager | keyword | John Doe | Assigned Manager for the user account. |
user.profile.employee_type | keyword | Regular | Further classification type for the user account. |
user.profile.job_family | keyword | 65-Sales | Job family associated with the user account. |
user.profile.job_family_group | keyword | GTM | Job family group associated with the user account. |
user.profile.management_level | keyword | Individual Contributor | If the user account is identified as a Manager or Individual contributor. |
user.profile.job_title | keyword | Field Sales | Job title assigned to the user account. |
user.profile.department | keyword | x256 | Department name associated with the user account. |
user.profile.organization | keyword | Elasticsearch Inc. | Organization name associated with the account. |
user.profile.location | keyword | US - Washington - Distributed | Assigned location for the user account. |
user.profile.mobile_phone | keyword | 222-222-2222 | |
user.profile.primaryPhone | keyword | 222-222-2222 | |
user.profile.secondEmail | keyword | first.l@elastic.co | Additional email addresses associated with the user account. |
user.profile.sup_org_id | keyword | SUP-ORG-75 | Primary organization ID for the user account. |
user.profile.supervisory_Org | keyword | Field Sales | Primary organization name for the user account. |
user.profile.assigned_mdm_id | keyword | 2950 | The primary host identifier (usually asset.id value) assigned to the user. This field acts as a correlation identifier for the host event document. |
user.account.create_date | date | June 5, 2023 @ 18:25:57.000 | Date account was created. |
user.account.activated_date | date | June 5, 2023 @ 18:25:57.000 | Date account was activated. |
user.account.change_date | date | June 5, 2023 @ 18:25:57.000 | Date user account record was last updated at source |
user.account.status.recovery | boolean | true/ false | A flag indicating if account is in recovery |
user.account.status.locked_out | boolean | true/ false | A flag indicating if account is currently locked out |
user.account.status.suspended | boolean | true/ false | A flag indicating if account has been suspended |
user.account.isAdmin | boolean | true/ false | A flag indicating if account is an Admin account |
user.account.isDelegatedAdmin | boolean | true/ false | A flag indicating if account has Delegated Admin rights |
user.account.isPriviledged | boolean | true/ false | A flag indicating if account is a Privileged account |
user.account.status.password_expired | boolean | true/ false | A flag indicating if account password has expired. |
user.account.status.deprovisioned | boolean | true/ false | A flag indicating if account has been deprovisioned |
user.account.password_change_date | date | June 5, 2023 @ 18:25:57.000 | Last date/time when account password was updated |
Field | Type | Generic Example | User Entity Example | Host Entity Example | Description |
---|---|---|---|---|---|
asset.category | keyword | - | Null | hardware | A further classification of the asset type beyond event.category. For example, for host assets {hardware, virtual, container, node}. For user assets {NULL ?} |
asset.type | keyword | - | Null | workstation | A sub classification of asset. For host assets {workstation, S3, Compute}. For user assets {NULL?}. |
asset.id | keyword | - | 00uhs72c27s6PiK7x1t7 | 2950 | A unique ID for the asset. For inventory integrations, it's the id generated from inventory data source. |
asset.name | keyword | - | Sourin Paul | Sourin Paul Macbook Pro | A common name for the asset. |
asset.vendor | keyword | - | - | Apple | Used primarily for 'Host' entities, the vendor name or brand associated with the asset. |
asset.product | keyword | - | - | MacBook Pro | Used primarily for 'Host' entities, the product name associated with the asset. |
asset.model | keyword | - | - | TBD | Used primarily for 'Host' entities, the model name or number associated with this asset. |
asset.version | keyword | - | - | TBD | Used primarily for 'Host' entities, the version or year associated with the asset. |
asset.owner | keyword | - | - | sourin.paul@elastic.co | The primary user entity identifier (usually an email address) who owns the 'Host' asset. |
asset.priority | keyword | Priority 1 | - | - | A priority classification for the asset obtained from outside the solution, such as from some external CMDB or Directory service. |
asset.criticality | keyword | Critical | - | - | A criticality classification obtained from outside the solution, such as from some external CMDB or Directory service. |
asset.business_unit | keyword | Analyst Experience | - | - | Business Unit associated with the asset (user or host). |
asset.costCenter | keyword | Security - Protections | - | - | Cost Center associated with the asset (user or host). |
asset.cost_center_hierarchy | keyword | Engineering | - | - | Additional cost center information associated with the asset (user or host). |
asset.status | keyword | ACTIVE | - | - | Current status of the asset in the inventory datasource. |
asset.last_status_change_date | date | June 5, 2023 @ 18:25:57.000 | - | - | The most recent date/time when the asset.status was updated. |
asset.create_date | date | June 5, 2023 @ 18:25:57.001 | - | - | For users, it's the hire date. For other assets, it's the in-service date. |
asset.end_date | date | June 5, 2023 @ 18:25:57.002 | - | - | For users, it's the termination date; for other assets, it's the out-of-service date. |
asset.first_seen | date | June 5, 2023 @ 18:25:57.003 | - | - | The first date/time the directory service or the security solution observed this asset. |
asset.last_seen | date | June 5, 2023 @ 18:25:57.004 | - | - | The most recent date/time the directory service or the security solution observed this asset. |
asset.last_updated | date | June 5, 2023 @ 18:25:57.005 | - | - | The most recent date/time this asset was updated in directory services. |
asset.serial_number | keyword | C02FG1G1MD6T | - | - | Serial number of the asset. |
asset.tags | keyword | watch, mdmaccess | - | - | Tags assigned at the MDM. |
asset.assigned_users | keyword | user1@email.com, user2@email.com | - | - | List of user ids (usually email addresses) assigned to the asset. The value from the asset.owner field should always be included. |
asset.assigned_users_are_admin | boolean | TRUE | - | - | Flag to identify if the assigned users have admin privileges. |
asset.is_managed | boolean | TRUE | - | - | If asset is managed by the organization. |
asset.last_enrolled_date | date | June 5, 2023 @ 18:25:57.005 | - | - | The most recent date/time the asset checked in with MDM. |
asset.data_classification | keyword | restricted | - | - | Data classification tier for the asset. |
asset.installed_extensions | keyword | Nested objects | List of installed extensions along with their metadata | ||
asset.installed_applications | keyword | Nested objects | List of installed applications along with their metadata |
- We have a set of risk.* fields in ECS that can be further nested under the asset.* object. Reference to Risk RFC.
Field | Type | Example | Description |
---|---|---|---|
os.build | keyword | 22F66 | Host OS Build information |
-
As part of Entity Analytics, we are ingesting metadata about Users and from various external vendor applications. We are storing all ingested metadata in Elasticsearch. After we map these fields to ECS, we will enrich these ingested events for risk-scoring scenarios (e.g., context enrichments) and detecting advanced analytics (UBA) use cases.
-
This schema will persist
Observed
(queried) entities from the ingested security log dataset in an Entity store. This entity store can be further extended to meet broader Asset Management needs. -
Additional enrichment use cases for existing prebuilt detection rules will leverage these ECS fields.
There are many sources of asset inventory repositories. In the mid-term, we are planning to ingest data from the following application providers:
- Azure Active Directory
- Active Directory DS
- Okta
- Workday
- GSuite
- GitHub
- Azure Active Directory
- Jamf
- Active Directory DS
- MS Intune
- ServiceNow Asset CMDB
-
Ingestion mechanisms: Entity Analytics fleet integrations are the primary ingesting mechanism for this dataset.
-
Usage mechanism: Elastic Security solution (Entity Analytics & Threat Hunting workflows) will be the primary user of the proposed ECS fields and values.
- We have a couple of fleet integrations under development. We want them to use these proposed ECS before being released.
- Schema/ field sets defined here focus on asset inventory data sources. Additional fields may need to be appended (ideally within this RFC lifecycle) to support the entity store needs.
- Due diligence is needed to avoid the proliferation of field sets and validate business requirements.
- In stage1, @jasonrhodes identified fields from o11y use cases and a potential conflict: #2215 (review)
The following are the people that consulted on the contents of this RFC.
- @sourinpaul | author
- @andrewkroh | subject matter expert
- @jamiehynds | subject matter expert
- @lauravoicu | subject matter expert
- @MikePaquette | subject matter expert
- @sourinpaul | sponsor
- ?
- Stage 0: #2215