From 52a8097a28637e32a4ad5d82193819313ba45bb6 Mon Sep 17 00:00:00 2001 From: Jurriaan Mous Date: Fri, 30 Aug 2024 13:06:03 +0200 Subject: [PATCH] Improve filters, key, protobuf and query documentation. --- core/documentation/filters.md | 61 +++--- core/documentation/key.md | 66 ++++--- core/documentation/protobuf.md | 38 ++-- core/documentation/query.md | 330 ++++++++++++++++++--------------- 4 files changed, 276 insertions(+), 219 deletions(-) diff --git a/core/documentation/filters.md b/core/documentation/filters.md index 14375b70a..03186acc4 100644 --- a/core/documentation/filters.md +++ b/core/documentation/filters.md @@ -1,16 +1,17 @@ # Filters -Filters can be applied to both [Get](query.md#get) and [Scan](query.md#scan) objects in -queries. Complex operations can be created by using the [`And`](#and) -and [`Or`](#or) filters. , and filters can be reversed using the [`Not`](#not) -filter. +Filters are useful tools that can be applied to both [Get](query.md#get) and [Scan](query.md#scan) objects in queries. +You can create complex operations by using the [`And`](#and) and [`Or`](#or) filters, while also having the ability to +reverse filters with the [`Not`](#not) filter. ## Reference Filters -The following filters operate on a property within a data object which is referred to with a + +The following filters operate on a property within a data object, which is identified by a [property reference](properties/references.md). ### Exists -Checks if a value exists. + +This filter checks whether a value exists. ```kotlin Exists( @@ -21,7 +22,8 @@ Exists( ``` ### Equals -Checks if a property reference's value is equal to the given value. + +Use this filter to check if a property reference's value is equal to a specified value. ```kotlin Equals( @@ -31,7 +33,8 @@ Equals( ``` ### GreaterThan -Checks if the referenced values are greater than the given value. + +This filter checks if the referenced values are greater than a specific value. ```kotlin GreaterThan( @@ -41,7 +44,8 @@ GreaterThan( ``` ### GreaterThanEquals -Checks if the referenced value is greater than or equal to the given value. + +Use this filter to check if the referenced value is greater than or equal to a specified value. ```kotlin GreaterThanEquals( @@ -51,7 +55,8 @@ GreaterThanEquals( ``` ### LessThan -Checks if the referenced value is less than the given value. + +This filter checks if the referenced value is less than a particular value. ```kotlin LessThan( @@ -61,7 +66,8 @@ LessThan( ``` ### LessThanEquals -Checks if the referenced value is less than or equal to the given value. + +Use this filter to check if the referenced value is less than or equal to a specified value. ```kotlin LessThanEquals( @@ -71,9 +77,11 @@ LessThanEquals( ``` ### Range -Checks if the referenced value is within the given range. + +This filter checks if the referenced value falls within a specified range. Maryk YAML: + ```kotlin Range( intPropertyReference with 2..42, @@ -86,7 +94,8 @@ Range( ``` ### Prefix -Checks if the referenced value is prefixed by the given value. + +Use this filter to check if the referenced value starts with a given prefix. ```kotlin Prefix( @@ -96,7 +105,8 @@ Prefix( ``` ### RegEx -Checks if the referenced value matches with the given regular expression. + +This filter checks if the referenced value matches a specified regular expression. ```kotlin RegEx( @@ -106,7 +116,8 @@ RegEx( ``` ### ValueIn -Checks if the referenced value is within the set of given values. + +Use this filter to check if the referenced value is included in a set of specified values. ```kotlin ValueIn( @@ -116,12 +127,14 @@ ValueIn( ``` ## Filter operations -Filter operations are powerful tools that allow you to construct complex queries. -They run on top of other filters, making it possible to create intricate and highly customizable search criteria. + +Filter operations provide powerful capabilities for constructing complex queries. They can run on top of other filters, +allowing you to create intricate and highly customizable search criteria. ### And -The And filter returns `true` if all the specified filters match. -This is an ideal filter to use if you want to find records that meet multiple conditions. + +The And filter returns `true` if all specified filters match. This filter is ideal when you want to find records that +meet multiple conditions. ```kotlin And( @@ -135,8 +148,9 @@ And( ``` ### Or -The Or filter returns `true` if one of the specified filters matches. -This is useful if you want to find records that match either of several conditions. + +The Or filter returns `true` if at least one of the specified filters matches. This is useful for finding records that +satisfy any of several conditions. ```kotlin Or( @@ -150,8 +164,9 @@ Or( ``` ### Not -The Not filter inverts the meaning of the specified filters. If multiple filters are passed, it performs an And operation. -This filter is useful if you want to exclude records that meet certain criteria. + +The Not filter inverts the meaning of specified filters. If multiple filters are provided, it performs an And operation. +This filter is beneficial for excluding records that meet specific criteria. ```kotlin Not( diff --git a/core/documentation/key.md b/core/documentation/key.md index 4565ca5ae..ef90daf99 100644 --- a/core/documentation/key.md +++ b/core/documentation/key.md @@ -1,45 +1,55 @@ # Keys in DataObjects -All DataObjects must be stored under a unique key, which acts as an index to the object. -The key serves as a permanent identifier for the object and can be used to retrieve it. -If you store new data under an existing key, the existing data will be overwritten. +All DataObjects must be stored under a unique key, which acts as an index to the object. The key serves as a permanent +identifier for the object and can be used to retrieve it. If you store new data under an existing key, the existing data +will be overwritten. ## Default UUID Keys -If you don't specify a key definition, the model will automatically generate 128-bit v4 UUID keys. -These keys are guaranteed to be unique, but they won't provide any benefits for scanning the data. +If you don't specify a key definition, the model will automatically generate 128-bit v4 UUID keys. These keys are +guaranteed to be unique, ensuring that no two DataObjects will share the same identifier. However, please note that +while they are unique, they do not provide any inherent benefits for scanning or ordering the data. ## Choosing the Right Key from the Start -It's essential to be mindful when designing the key for a DataModel, as the -key structure cannot be changed after data has been stored (without complex migrations). -Consider the primary use case and ordering of the data and make sure the key is optimized -for this purpose. Secondary use cases can be addressed by adding an index. +It's essential to be mindful when designing the key for a DataModel since the key structure cannot be changed after data +has been stored (without complex migrations). Consider the primary use case and the desired ordering of the data +carefully. Make sure the key is optimized for this purpose upfront. If secondary use cases arise, they can often be +addressed by adding an index later. ## Properties for Key Structure -Key structures must have fixed byte lengths. This way the location of key elements are -predictable which are beneficial for scans over keys. +Key structures must have fixed byte lengths. This predictability is crucial as it allows for efficient indexing and +scanning over keys. -Properties that can be used for key elements include numbers, dates and times, fixed bytes, references, enums, booleans, -multi-type objects, and ValueDataModels containing similar values. +Properties that can be utilized for key elements include numbers, dates and times, fixed bytes, references, enums, +booleans, multi-type objects, and ValueDataModels containing similar values. -Properties that cannot be used in keys include strings, flexible bytes, sets, lists, maps, and -embedded models, as they have varying byte lengths. +On the other hand, properties that cannot be used in keys include strings, flexible bytes, sets, lists, maps, and +embedded models, as they possess varying byte lengths, which disrupts the consistency required for effective key +structures. -## The order of keys -Keys are stored in order, which means that data scans will traverse or skip -the data in the same order. If the key starts with a reference, the data scan -can start at the exact location for that reference. +## The Order of Keys -If the data is often requested in the newest-first order, it is recommended to -reverse the date of creation so that new data is retrieved first. +Keys are stored in a specific order, meaning that data scans will traverse or skip the data in the same organized +manner. If the key begins with a reference, scans can efficiently start at the exact location corresponding to that +reference. -## Tips on designing a key +If the data is often requested in a newest-first order, it is advisable to reverse the date of creation in the key +structure. This way, newer data will be retrieved first during scans, enhancing the user experience. -- Consider performance: The key structure should be optimized for the most common use cases, as it will affect the performance of data retrieval and scans. -- If data "belongs" to a particular entity or person, start the key with a reference to that entity or person. -- If data needs to be ordered by time, include the time in the key. If newer data is frequently requested first, reverse the time. -- If the data has a primary multi-type property, include the type ID in the key so you can quickly retrieve data objects of a specific type. If time is also included in the key, make sure to place it after the type ID so that data is still ordered by time. -- If date precision in nanoseconds is somehow not enough, consider adding a random number to the key. -- Use indexing: If you have a key structure that is not optimized for your use case, you can use an index to improve performance. This is especially useful if you have large datasets. +## Tips on Designing a Key + +- **Consider performance:** The key structure should be optimized for the most common use cases, as this will directly + impact the performance of data retrieval and scans. +- **Entity association:** If data "belongs" to a particular entity or person, start the key with a reference to that + entity or person. This associativity can streamline searches. +- **Time ordering:** If data needs to be ordered by time, include the time in the key. If newer data is frequently + requested first, reverse the time to prioritize its retrieval. +- **Type identification:** If the data has a primary multi-type property, include the type ID in the key to enable quick + retrieval of data objects of a specific type. Ensure that any time information follows the type ID to maintain + chronological order. +- **Date precision:** If date precision in nanoseconds is insufficient for your application's needs, consider appending + a random number to the key for additional uniqueness. +- **Use indexing:** If you find that your key structure is not optimized for your primary use case, utilize indexing to + improve performance. This is especially beneficial when dealing with large datasets. diff --git a/core/documentation/protobuf.md b/core/documentation/protobuf.md index 45363f557..5909ca626 100644 --- a/core/documentation/protobuf.md +++ b/core/documentation/protobuf.md @@ -1,24 +1,34 @@ # Protobuf Transportation The encoding standard for [ProtoBuf V3](https://developers.google.com/protocol-buffers/) has been adopted for efficient -and compact transportation of data. Developed by Google, ProtoBuf is a widely adopted standard. Currently, only the -encoding standard has been adopted, and schema generation is yet to be implemented. +and compact transportation of data. Developed by Google, ProtoBuf is a widely adopted standard used across various +platforms and languages for serializing structured data. Currently, only the encoding standard has been implemented, +while schema generation is yet to be developed. -For a more in-depth understanding of how values are encoded, refer to the [ProtoBuf encoding documentation](https://developers.google.com/protocol-buffers/docs/encoding) +For a more in-depth understanding of how values are encoded, refer to +the [ProtoBuf encoding documentation](https://developers.google.com/protocol-buffers/docs/encoding), which provides +detailed insights on the encoding mechanisms and examples. -## Key Value pairs +## Key Value Pairs -A ProtoBuf message is built using key-value pairs, where the key contains a tag that identifies the encoded property and -a wire_type that indicates the type of value that was encoded. The value is encoded in the byte format for transport, -and the encoding format for each property type is documented in the [properties documentation](properties/properties.md). +A ProtoBuf message is constructed using key-value pairs. In this structure, the key consists of a tag that uniquely +identifies the encoded property, along with a wire type that specifies the type of value being encoded. The actual value +is then represented in byte format for efficient transport. The encoding format for each property type is thoroughly +documented in the [properties documentation](properties/properties.md), which serves as a reference for developers +looking to implement and understand the specifics of encoding different data types. ## Wire Types -Maryk supports all wire types supported by ProtoBuf, including: +Maryk supports all wire types defined by ProtoBuf, including: -* VarInt: A variable integer used for numeric values that grow in size with the value. -* Length Delimited: Used for variable length values. The length of the bytes is preceded by the value. - It can also contain key-value pairs of embedded messages. -* 32 Bit: Used for values of 4 bytes. -* 64 Bit: Used for values of 8 bytes. -* Start Group / End Group: Not currently used and also deprecated in ProtoBuf. +* **VarInt**: A variable-length integer used for numeric values, allowing for efficient storage of small values while + accommodating larger integers without wasting space. +* **Length Delimited**: This type is utilized for values that can vary in length. The actual bytes of the value are + prefixed by a length field, making it versatile for use with strings and byte arrays. Additionally, it can encapsulate + key-value pairs of embedded messages. +* **32 Bit**: Specifically employed for values that are exactly 4 bytes in size, commonly used for fixed-width numerical + types. +* **64 Bit**: Designed for values that are exactly 8 bytes, typically used for large numerical types or higher-precision + floating-point numbers. +* **Start Group / End Group**: These wire types are currently not in use and are also deprecated in the ProtoBuf + specification. It is advisable to avoid using them in new implementations. diff --git a/core/documentation/query.md b/core/documentation/query.md index cfb6fd015..a5a1d19c5 100644 --- a/core/documentation/query.md +++ b/core/documentation/query.md @@ -1,30 +1,33 @@ # Query Data -You can perform a selection of query actions on a store of data objects. This store can be local or the -query action can be serialized and send to a store on another server to be performed there. +You can perform a selection of query actions on a store of data objects. This store can be local, or the +query action can be serialized and sent to a store on another server to be performed there. ## Basic Query Actions -Maryk provides some basic query actions which are basically based on the CRUD model: you can add, change, get, -scan or delete objects. +Maryk provides some basic query actions which are fundamentally based on the CRUD model: you can add, change, get, +scan, or delete objects. -The Get and Scan request types are available in 3 flavors: -- `GetRequest`/`ScanRequest` - Returns `Values` to represent the object as it is on requested time. +The **Get** and **Scan** request types are available in three variations: + +- `GetRequest`/`ScanRequest` - Returns `Values` to represent the object as it is at the requested time. - `GetUpdatesRequest`/`ScanUpdatesRequest` - Returns `Updates` like `Addition`/`Change`/`Removal` to show - the changing of data and its view defined by filters and limit/offset over time. Is possible to use in `executeFlow` - to get live updates as new data is processed. -- `GetChangesRequest`/`ScanChangesRequest` - Returns any data as versioned changes as it is stored in the datastore - ordered per data object. + the changing of data and its view defined by filters and limit/offset over time. It is possible to use this in + `executeFlow` + to get live updates as new data is processed. +- `GetChangesRequest`/`ScanChangesRequest` - Returns any data as versioned changes as they are stored in the datastore + ordered per data object. ### Add -With [`AddRequest`](../src/commonMain/kotlin/maryk/core/query/requests/AddRequest.kt) objects can be -added to a store. When applied it will deliver an [`AddResponse`](../src/commonMain/kotlin/maryk/core/query/responses/AddResponse.kt) -with a status on each object to add. +With [`AddRequest`](../src/commonMain/kotlin/maryk/core/query/requests/AddRequest.kt), objects can be +added to a store. When applied, it will deliver an [`AddResponse`](../src/commonMain/kotlin/maryk/core/query/responses/AddResponse.kt) +with a status on each object added. -Example: +**Example:** Kotlin: + ```kotlin val addRequest = Person.add( Person.run { create( @@ -40,17 +43,18 @@ val addRequest = Person.add( ### Change -With [`ChangeRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ChangeRequest.kt) objects can be -changed in a store. When applied it will deliver an [`ChangeResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ChangeResponse.kt) +With [`ChangeRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ChangeRequest.kt), objects can be +modified in a store. When applied, it will deliver an [`ChangeResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ChangeResponse.kt) with a status on each change. Refer to [property operations](properties/operations.md) to see how to apply changes to properties. -Example: +**Example:** + ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change +val person1Key // Key of person 1 to change +val person2Key // Key of person 2 to change val changeRequest = Person.change( person1Key.change( @@ -64,62 +68,67 @@ val changeRequest = Person.change( ``` Simplify references with Kotlin `run{}`: + ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change +val person1Key // Key of person 1 to change +val person2Key // Key of person 2 to change val changeRequest = Person.run { - change( - person1Key.change( - Check(ref { firstName } with "Jane"), - Change(ref { lastName } with "Doe") - ), - person2Key.change( - Change(ref { lastName } with "Smith") - ) - ) + change( + person1Key.change( + Check(ref { firstName } with "Jane"), + Change(ref { lastName } with "Doe") + ), + person2Key.change( + Change(ref { lastName } with "Smith") + ) + ) } ``` ### Delete -With [`DeleteRequest`](../src/commonMain/kotlin/maryk/core/query/requests/DeleteRequest.kt) -objects can be deleted inside a store. The objects can be deleted by passing + +With [`DeleteRequest`](../src/commonMain/kotlin/maryk/core/query/requests/DeleteRequest.kt), +objects can be deleted from a store. The objects can be deleted by passing a list of object [`keys`](key.md). Objects can either be hard or soft deleted. -With a hard delete the data is certainly gone, and with soft delete (Default) it is -still in the store but not viewable unless specifically requested. -When applied it will deliver an [`DeleteResponse`](../src/commonMain/kotlin/maryk/core/query/responses/DeleteResponse.kt) -with a status on each delete. +With a hard delete, the data is permanently removed; with a soft delete (the default), it remains in the store but is +not viewable unless specifically requested. +When applied, it will deliver an [`DeleteResponse`](../src/commonMain/kotlin/maryk/core/query/responses/DeleteResponse.kt) +with a status on each deletion. + +**Example:** -Example: ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change +val person1Key // Key of person 1 to change +val person2Key // Key of person 2 to change -val changeRequest = Person.delete( +val deleteRequest = Person.delete( person1Key, person2Key, hardDelete = true ) ``` -### Get -With [`GetRequest`](../src/commonMain/kotlin/maryk/core/query/requests/GetRequest.kt) -multiple specific objects can be queried by their [key](key.md). -To select a subset of values in the query use `select`. -It is possible to filter the results with [filters](filters.md), order the results or to include -soft deleted results by passing `filterSoftDeleted=false`. +### Get -It is also possible to view the objects at a certain version with `toVersion` +With [`GetRequest`](../src/commonMain/kotlin/maryk/core/query/requests/GetRequest.kt), +multiple specific objects can be queried by their [key](key.md). +To select a subset of values in the query, use `select`. +It is possible to filter the results with [filters](filters.md), order the results, or include +soft-deleted results by passing `filterSoftDeleted=false`. + +You can also view the objects at a certain version with `toVersion` if the store supports viewing past versions. -When applied it will deliver an [`ValuesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ValuesResponse.kt) -with a list with [`ValuesWithMetaData`](../src/commonMain/kotlin/maryk/core/query/ValuesWithMetaData.kt) -containing the `key`, `object`, `firstVersion`, `lastVersion` and `isDeleted`. +When applied, it will deliver an [`ValuesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ValuesResponse.kt) +with a list of [`ValuesWithMetaData`](../src/commonMain/kotlin/maryk/core/query/ValuesWithMetaData.kt) +containing the `key`, `object`, `firstVersion`, `lastVersion`, and `isDeleted`. + +**Example:** -Example: ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change +val person1Key // Key of person 1 to change +val person2Key // Key of person 2 to change val getRequest = Person.get( person1Key, @@ -128,41 +137,43 @@ val getRequest = Person.get( ``` With all parameters: + ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change - -val getRequest = Person.run{ - get( - person1Key, - person2Key, - where = And( - Equals(ref { firstName } with "Clark"), - Exists(ref { lastName }) - ), - order = ref { lastName }.ascending(), - toVersion = 2L, - filterSoftDeleted = false - ) +val person1Key // Key of person 1 to change +val person2Key // Key of person 2 to change + +val getRequest = Person.run { + get( + person1Key, + person2Key, + where = And( + Equals(ref { firstName } with "Clark"), + Exists(ref { lastName }) + ), + order = ref { lastName }.ascending(), + toVersion = 2L, + filterSoftDeleted = false + ) } ``` ### Scan -With [`ScanRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ScanRequest.kt) + +With [`ScanRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ScanRequest.kt), multiple objects can be queried by passing a startKey to scan from and filters on key parts to end it. -To select a subset of values in the query use `select` -It is possible to filter the results with [filters](filters.md), order or limit the results (default= 100). -It is also possible to include soft deleted results by passing `filterSoftDeleted=false`. +To select a subset of values in the query, use `select`. +It is possible to filter the results with [filters](filters.md), order, or limit the results (default= 100). +You can also include soft-deleted results by passing `filterSoftDeleted=false`. -It is also possible to view the objects at a certain version with `toVersion` +Additionally, you can view the objects at a certain version with `toVersion` if the store supports viewing past versions. -When applied it will deliver an [`ValuesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ValuesResponse.kt) -with a list with [`ValuesWithMetaData`](../src/commonMain/kotlin/maryk/core/query/ValuesWithMetaData.kt) -containing the `key`, `object`, `firstVersion`, `lastVersion` and `isDeleted`. +When applied, it will deliver an [`ValuesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ValuesResponse.kt) +with a list of [`ValuesWithMetaData`](../src/commonMain/kotlin/maryk/core/query/ValuesWithMetaData.kt) +containing the `key`, `object`, `firstVersion`, `lastVersion`, and `isDeleted`. ```kotlin -val timedKey // Key which start at certain time +val timedKey // Key starting at a certain time val scanRequest = Logs.scan( startKey = timedKey @@ -170,8 +181,9 @@ val scanRequest = Logs.scan( ``` With all parameters: + ```kotlin -val timedKey // Key which start at certain time +val timedKey // Key starting at a certain time val scanRequest = Logs.run { scan( @@ -188,27 +200,30 @@ val scanRequest = Logs.run { ``` ### Scan/Get Updates -You can request updates on Objects ordered by version with -[`GetUpdatesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/GetUpdatesRequest.kt). + +You can request updates on objects ordered by version with +[`GetUpdatesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/GetUpdatesRequest.kt) or [`ScanUpdatesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ScanUpdatesRequest.kt). `maxVersions` (default=1) can be used to control how many versions are returned -at maximum. To return more than 1 version the DataStore needs to have `keepAllVersions` set to `true`. +at maximum. To return more than one version, the DataStore needs to have `keepAllVersions` set to `true`. -When applied it will deliver an [`UpdatesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/UpdatesResponse.kt) -with a list with [`IsUpdatesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/updates/IsUpdateResponse.kt) -which are either `AdditionUpdate`, `ChangeUpdate`, `RemovalUpdate` and always starts with `OrderedKeysUpdate` with the initial -ordering of keys. +When applied, it will deliver an [`UpdatesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/UpdatesResponse.kt) +with a list of [`IsUpdatesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/updates/IsUpdateResponse.kt) +which are either `AdditionUpdate`, `ChangeUpdate`, `RemovalUpdate`, and always starts with `OrderedKeysUpdate`, +reflecting the initial ordering of keys. This request type can also be used inside `executeFlow` to listen to updates as they happen in the data store. -Optionally it is possible to pass an `orderedKeys` parameter to a `ScanUpdatesRequest`. It will then pass any changes to -that list as well. This way you are certain you receive hard deleted values, or added values which had their values changed, -so they are within range of the passed order/limit. +Optionally, you can pass an `orderedKeys` parameter to a `ScanUpdatesRequest`. This allows you to receive any changes to +that list as well. This way, you are assured to receive hard-deleted values or added values that had their values +changed, +ensuring they are within range of the passed order/limit. + +**Get updates with all parameters:** -Get changes with all parameters: ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change +val person1Key // Key of person 1 to change +val person2Key // Key of person 2 to change val getRequest = Person.run { getUpdates( @@ -226,16 +241,17 @@ val getRequest = Person.run { ), order = ref { lastName }.ascending(), toVersion = 2000L, - filterSoftDeleted = false, + filterSoftDeleted = false, fromVersion = 1000L, maxVersions = 100 ) } ``` -Scan updates with all parameters: +**Scan updates with all parameters:** + ```kotlin -val timedKey // Key which start at certain time +val timedKey // Key starting at a certain time val scanRequest = Logs.scanUpdates( startKey = timedKey, @@ -246,83 +262,89 @@ val scanRequest = Logs.scanUpdates( message ) }, - where = GreaterThanEquals( - Logs { severity::ref } with Severity.ERROR - ), - order = ref { timeStamp }.descending(), - filterSoftDeleted = false, - limit = 50, - fromVersion = 1000L, - toVersion = 2000L, - maxVersions = 100, - orderedKeys = listOf(log1Key, log2Key, log3Key) + where = GreaterThanEquals( + Logs { severity::ref } with Severity.ERROR + ), + order = ref { timeStamp }.descending(), + filterSoftDeleted = false, + limit = 50, + fromVersion = 1000L, + toVersion = 2000L, + maxVersions = 100, + orderedKeys = listOf(log1Key, log2Key, log3Key) ) ``` ### Scan/Get Changes -You can request all the changes on Objects ordered by version with -[`GetChangesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/GetChangesRequest.kt). -or [`ScanChangesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ScanChangesRequest.kt). -`maxVersions` (default=1) can be used to control how many versions are returned -at maximum. To return more than 1 version the DataStore needs to have `keepAllVersions` set to `true`. -When applied it will deliver an [`ChangesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ChangesResponse.kt) -with a list with [`DataObjectVersionedChange`](../src/commonMain/kotlin/maryk/core/query/changes/DataObjectVersionedChange.kt) -containing the `key` and `changes` with a list of objects containing the version and changes. +You can request all the changes on Objects ordered by version using two methods: +[`GetChangesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/GetChangesRequest.kt) and [ +`ScanChangesRequest`](../src/commonMain/kotlin/maryk/core/query/requests/ScanChangesRequest.kt). + +The `maxVersions` parameter (default=1) controls how many versions are returned at maximum. To return more than one +version, ensure that the DataStore has `keepAllVersions` set to `true`. -NOTE: This type cannot have a filter or order on mutable properties since then the changes could lead to unreliable results. -For example if a value is modified it could lead to a different position in the ordering or can lead to a where filter to -filter the result away. Use Get/Scan Updates to see changes in these cases. +When applied, these methods will deliver a [ +`ChangesResponse`](../src/commonMain/kotlin/maryk/core/query/responses/ChangesResponse.kt), which includes a list of [ +`DataObjectVersionedChange`](../src/commonMain/kotlin/maryk/core/query/changes/DataObjectVersionedChange.kt). This +response contains details such as the `key` and `changes`, which provide insights into the version and any modifications +made. + +**NOTE:** This type cannot have filters or orders on mutable properties, as doing so may yield unreliable results. For +instance, if a value is modified, it could affect its ordering position or inadvertently filter out results. In such +cases, consider using Get/Scan Updates to track changes effectively. Get changes with all parameters: + ```kotlin -val person1Key // Containing the key of person 1 to change -val person2Key // Containing the key of person 2 to change +val person1Key // Key representing person 1 whose data will be changed. +val person2Key // Key representing person 2 whose data will be changed. val getRequest = Person.run { - getChanges( - person1Key, - person2Key, - select = graph { - listOf( - firstName, - lastName - ) - }, - where = And( - Equals(ref { firstName } with "Clark"), - Exists(ref { lastName }) - ), - order = ref { lastName }.ascending(), - toVersion = 2000L, - filterSoftDeleted = false, - fromVersion = 1000L, - maxVersions = 100 - ) + getChanges( + person1Key, + person2Key, + select = graph { + listOf( + firstName, + lastName + ) + }, + where = And( + Equals(ref { firstName } with "Clark"), + Exists(ref { lastName }) + ), + order = ref { lastName }.ascending(), + toVersion = 2000L, + filterSoftDeleted = false, + fromVersion = 1000L, + maxVersions = 100 + ) } ``` Scan changes with all parameters: + ```kotlin -val timedKey // Key which start at certain time +val timedKey // Key that indicates the starting point for scanning at a certain time. val scanRequest = Logs.scanChanges( - startKey = timedKey, - select = graph { - listOf( - timeStamp, - severity, - message - ) - }, - where = GreaterThanEquals( - Logs { severity::ref } with Severity.ERROR - ), - order = ref { timeStamp }.descending(), - filterSoftDeleted = false, - limit = 50, - fromVersion = 1000L, - toVersion = 2000L, - maxVersions = 100 + startKey = timedKey, + select = graph { + listOf( + timeStamp, + severity, + message + ) + }, + where = GreaterThanEquals( + Logs { severity::ref } with Severity.ERROR + ), + order = ref { timeStamp }.descending(), + filterSoftDeleted = false, + limit = 50, + fromVersion = 1000L, + toVersion = 2000L, + maxVersions = 100 ) ```