How to do CRUD

[viktorklang]

Note that what is meant by CRUD is not SQL, or joins, but rather being able to get an Entity value, modifying it, and having the modified version stored for the next command/request. So "destructive updates".

Could in theory be implemented on top of the EventSourcing support by either storing the new Entity value as an event, or by repeatedly generating new Snapshots for each new state.

This also impacts the user-facing API as they would not have to deal with anything but the inbound commands (not events).

[andyczerwonka]

I think this would be a simplified API, but it would also encourage much larger message sizes (full entity/graph) which would have a negative impact on performance and storage requirements.

[viktorklang]

@andyczerwonka That's not necessarily a problem, since snapshotting for event-sourced entities effectively also is the entire entity graph. Or rather, if that is a problem, then the problem is rather general for all state storage.
The real benefit for Cloudstate is that what we store is effectively a blob, ans as such, we can compress/decompress the data generically.

[andyczerwonka]

@viktorklang agree, in the general case this CRUD story could be achieved purely by using a snapshot for every event. I maintain that, in practice, this could be a problem so I would encourage event sourcing, but for the simple case it'd work well in that it'd simplify the read case.

[viktorklang]

@andyczerwonka Agreed, these tradeoffs need to be clearly documented. :)

[ralphlaude]

@viktorklang,
This is an extented implementation i propose based on (https://github.com/cloudstateio/cloudstate/pull/143/files of @viktorklang).

What i did:

• Introduce a new class io.cloudstate.javasupport.crud.KeyValue.ChangedMap for dealing with last changed (updates and remove) in the CRUD entity.
  This is useful for CQRS
• I change the definition of the io.cloudstate.javasupport.crud.CrudEntity annotation and also the logic for CloudState.registerCrudEntity

What i would like to do:

• Who should register the key value proto file descriptor io.cloudstate.keyvalue.KeyValue.getDescriptor()?
  i would propose to do in CloudState.registerCrudEntity so it is implicit to the user
• How to define persistent actor id for the CRUD entity? The CRUD entity is backed by a persistent actor and this actor should receive all request
  for the CRUD entity. Unfortunately the id (entity_key) for this actor cannot be extract from the GRPC request paylod, the entity_key should be created upfront.
  I would suggest to introduce a new entity type that can be passed to io.cloudstate.javasupport.impl.eventsourced.EventSourcedStatefulService and
  further down to io.cloudstate.proxy.EntityUserFunctionTypeSupport which can decide how to create the entity_key of persistent actor.

What are the next steps:

• better naming, better documentation and better code design
• how to deal with the CRUD entity state in the io.cloudstate.javasupport.eventsourced.CommandContext to be able to ctx.setState like @viktorklang proposed?
• how to deal with sub-aggregates regarding the key? Use the same key or not for saving sub-aggregates?

Any comments are welcome.

[ralphlaude]

@viktorklang,
I am lost in exploration :).

I would like to know if Cloudstate still wants to support CRUD. If it is the case I would also like to know at least one use case to address and which path we eventually want to go.

I am evaluating doing CRUD on top of Event Sourcing (using the shopping cart example) and the only option I see now is based on Key Value. In this scenario the CRUD entity is a collection of key value. Perhaps there are another options and I would be happy to know about and to explore on them.

I would be glad to have some feedback here :).

[viktorklang]

@ralphlaude You definitely deserve some feedback here—it's just been crazy busy over here for a while. I'll try to sink some time into this PR tomorrow! :) (Thanks for working on this!!)

[ralphlaude]

@viktorklang don't worry everything is fine :), take your time. I just want to make sure things are going the right way.

[ralphlaude]

@viktorklang,
ping!

in the context of CRUD we have the CRUD entity (having the entity_key) as aggregate with multiples sub-aggregates.
For accessing the CRUD entity we have to use the entity_key and for accessing any sub-aggregate we have to use the entity_key for the CRUD entity.
So if we load any sub-aggregate in the stateful function we will mess up the keys. I don't see how to do this without messing up the keys
(We cannot directly access any sub-aggregate).

We have some options to convey commands from the stateful function to the CRUD entity:

• The entity_key for the CRUD entity could be created transparently by cloudstate
• The command should be extended with the key for the corresponding sub-aggregate
• Define CRUD command in the cloudstate protocol, such a command will have the mandatory fields entity_key and sub_entity_key.

What should be the option to choose? Are there another options?

[ralphlaude]

@viktorklang,

for a CRUD entity we would like the command context to do operations like setState, getState or even deserialize.

The existing command context for event sourced stateful function can be extend with those functionalities and it is the easy way to go.

The other way to is to have a specific command context for CRUD entity with those functionalities. In this case we should know when to create each of them and for that we want to know which kind of event sourced entity we want to create. Here we have CRUD entity and EventSourced entity which are both EventSourced entity. This option could be more complicated because we have to make a difference between two differents EventSourced entity and we could eventually add a new kind of stateful service (CRUD stateful service) in the protocol.

What do you think? Are there another options?

[viktorklang]

@ralphlaude I think it would be expected to have create/load/save/delete operations.
I think it is worth splitting this "task" up into two distinct parts: the protocol part, and the UX part.

[ralphlaude]

@viktorklang, it makes sense and i will focus first on the protocol part.

[ralphlaude]

@viktorklang,
A possible CRUD grpc protocol which is in the crud_two.proto and all the implementation for the first draft are in the packages
io.cloudstate.javasupport.crudtwo, io.cloudstate.javasupport.impl.crudtwo and io.cloudstate.proxy.crudtwo.
There is an example here io.cloudstate.javasupport.crudtwo.CrudEntityExample.
What i did:

• Defining a crud service with the operations create, fetch, update and delete.
• Only the operations create an fetch are implemented to show how it will work.
• The java support is also partly implemented and will be improved.
• The entity is also implemented in the proxy and will be improved.

This is my proposal and any comments are welcome.

It can be seen here:

• https://github.com/ralphlaude/cloudstate/tree/prototype-crud-on-event-sourcing/java-support/src/main/scala/io/cloudstate/javasupport/impl/crudtwo
• https://github.com/ralphlaude/cloudstate/tree/prototype-crud-on-event-sourcing/java-support/src/main/java/io/cloudstate/javasupport/crudtwo
• https://github.com/ralphlaude/cloudstate/tree/prototype-crud-on-event-sourcing/proxy/core/src/main/scala/io/cloudstate/proxy/crudtwo

[viktorklang]

Pinging @pvlugter @jroper :)

[ralphlaude]

Hi,
everything is now on this branch (https://github.com/ralphlaude/cloudstate/tree/prototype-crud-on-event-sourcing) in the dedicated package crud. This is more explicit (master...ralphlaude:prototype-crud-on-event-sourcing). There is only one package for crud now.

[ralphlaude]

@viktorklang, @pvlugter, @jroper,
i think i have a version you can take a look on. The version defines the protocol with snapshot and also implements it. Please take a look and give some feedback. After that i will go a step further with testing and the documentation.
Here - master...ralphlaude:prototype-crud-on-event-sourcing

[ralphlaude]

The progress can be seen here (master...ralphlaude:prototype-crud-on-event-sourcing) and is as follow:

• the GRPC protocol is defined and implemented see below
• the java support for CRUD is defined and implemented
• the CRUD service can be registered as service in cloudstate
• the CRUD entity is defined and implemented in the proxy
• the CRUD service support snapshotting
• there is a shopping cart example in the java sample that should be completed

The CRUD service in the java-support is right now initialized when a command come in. When the service starts there is not information upfront for initializing the CRUD service until a command is sent. I am wondering if there is another way to do the initialization upfront.

Each command has a type so we know which service operation to route the command to.

syntax = "proto3";

import "google/protobuf/descriptor.proto";

package cloudstate;

option java_package = "io.cloudstate";
option go_package = "github.com/cloudstateio/go-support/cloudstate/;cloudstate";

extend google.protobuf.FieldOptions {
    bool crud_command_type = 50005;
}

Here is the content of the crud.proto file which defines the CRUD service grpc protocol:

syntax = "proto3";

package cloudstate.crud;

// Any is used so that domain events defined according to the functions business domain can be embedded inside
// the protocol.
import "google/protobuf/any.proto";
import "google/protobuf/empty.proto";
import "cloudstate/entity.proto";

option java_package = "io.cloudstate.protocol";
option go_package = "cloudstate/protocol";

// The type of the command to be executed
enum CrudCommandType {
    UNKNOWN = 0;
    CREATE = 1;
    FETCH = 2;
    FETCHALL = 3;
    UPDATE = 4;
    DELETE = 5;
}

// The persisted state with the sequence number of the last snapshot
message CrudState {
    // The state payload
    google.protobuf.Any payload = 2;

    // The sequence number when the snapshot was taken.
    int64 snapshot_sequence = 1;
}

// Message for initiating the command execution
// which contains the command type to be able to identify the crud operation being called
message CrudEntityCommand {
    // The ID of the entity.
    string entity_id = 1;

    // The ID of a sub entity.
    string sub_entity_id = 2;

    // Command name
    string name = 3;

    // The command payload.
    google.protobuf.Any payload = 4;

    // The command type.
    CrudCommandType type = 5;
}

// The command to be executed which can be for any of the supported
// (create, fetch, save, delete, fetchAll) crud operations.
message CrudCommand {
    // The name of the service this crud entity is on.
    string service_name = 1;

    // The ID of the entity.
    string entity_id = 2;

    // The ID of a sub entity.
    string sub_entity_id = 3;

    // A command id.
    int64 id = 4;

    // Command name
    string name = 5;

    // The command payload.
    google.protobuf.Any payload = 6;

    // The persisted state to be conveyed between persistent entity and the user function.
    CrudState state = 7;
}

// A reply to a command.
message CrudReply {

    // The id of the command being replied to. Must match the input command.
    int64 command_id = 1;

    // The action to take
    ClientAction client_action = 2;

    // Any side effects to perform
    repeated SideEffect side_effects = 3;

    // An optional state to persist.
    google.protobuf.Any state = 4;

    // An optional snapshot to persist. It is assumed that this snapshot will have
    // the state of any events in the events field applied to it. It is illegal to
    // send a snapshot without sending any events.
    google.protobuf.Any snapshot = 5;
}

// A reply message type for the gRPC call.
message CrudReplyOut {
    oneof message {
        CrudReply reply = 1;
        Failure failure = 2;
    }
}

// The CRUD Entity service
// Provides read and write operations for managing the entity state.
// For a CRUD entity an event represents also the whole state of the entity.
// A read operation only transport the state from the entity without changing it.
// Typical read operations are fetch and fetchAll.
// A typical write operation might transform the state of the entity and crate a new one.
// Typical write operations are create, save, and delete.
// Each write operation may generate zero or one event which is then sent to the entity.
// The entity is expected to apply these event to its state.
service Crud {

    // Create a sub entity.
    rpc create(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of a sub entity.
    rpc fetch(CrudCommand) returns (CrudReplyOut) {}

    // Save a updated sub entity.
    rpc save(CrudCommand) returns (CrudReplyOut) {}

    // Delete a sub entity.
    rpc delete(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of the whole entity.
    rpc fetchAll(CrudCommand) returns (CrudReplyOut) {}
}

[ralphlaude]

@viktorklang, @jroper, @pvlugter here the appropriate PR - #220

[sleipnir]

The progress can be seen here (master...ralphlaude:prototype-crud-on-event-sourcing) and is as follow:

* the GRPC protocol is defined and implemented see below

* the java support for CRUD is defined and implemented

* the CRUD service can be registered as service in cloudstate

* the CRUD entity is defined and implemented in the proxy

* the CRUD service support snapshotting

* there is a shopping cart example in the java sample that should be completed

The CRUD service in the java-support is right now initialized when a command come in. When the service starts there is not information upfront for initializing the CRUD service until a command is sent. I am wondering if there is another way to do the initialization upfront.

Each command has a type so we know which service operation to route the command to.

syntax = "proto3";

import "google/protobuf/descriptor.proto";

package cloudstate;

option java_package = "io.cloudstate";
option go_package = "github.com/cloudstateio/go-support/cloudstate/;cloudstate";

extend google.protobuf.FieldOptions {
    bool crud_command_type = 50005;
}

Here is the content of the crud.proto file which defines the CRUD service grpc protocol:

syntax = "proto3";

package cloudstate.crud;

// Any is used so that domain events defined according to the functions business domain can be embedded inside
// the protocol.
import "google/protobuf/any.proto";
import "google/protobuf/empty.proto";
import "cloudstate/entity.proto";

option java_package = "io.cloudstate.protocol";
option go_package = "cloudstate/protocol";

// The type of the command to be executed
enum CrudCommandType {
    UNKNOWN = 0;
    CREATE = 1;
    FETCH = 2;
    FETCHALL = 3;
    UPDATE = 4;
    DELETE = 5;
}

// The persisted state with the sequence number of the last snapshot
message CrudState {
    // The state payload
    google.protobuf.Any payload = 2;

    // The sequence number when the snapshot was taken.
    int64 snapshot_sequence = 1;
}

// Message for initiating the command execution
// which contains the command type to be able to identify the crud operation being called
message CrudEntityCommand {
    // The ID of the entity.
    string entity_id = 1;

    // The ID of a sub entity.
    string sub_entity_id = 2;

    // Command name
    string name = 3;

    // The command payload.
    google.protobuf.Any payload = 4;

    // The command type.
    CrudCommandType type = 5;
}

// The command to be executed which can be for any of the supported
// (create, fetch, save, delete, fetchAll) crud operations.
message CrudCommand {
    // The name of the service this crud entity is on.
    string service_name = 1;

    // The ID of the entity.
    string entity_id = 2;

    // The ID of a sub entity.
    string sub_entity_id = 3;

    // A command id.
    int64 id = 4;

    // Command name
    string name = 5;

    // The command payload.
    google.protobuf.Any payload = 6;

    // The persisted state to be conveyed between persistent entity and the user function.
    CrudState state = 7;
}

// A reply to a command.
message CrudReply {

    // The id of the command being replied to. Must match the input command.
    int64 command_id = 1;

    // The action to take
    ClientAction client_action = 2;

    // Any side effects to perform
    repeated SideEffect side_effects = 3;

    // An optional state to persist.
    google.protobuf.Any state = 4;

    // An optional snapshot to persist. It is assumed that this snapshot will have
    // the state of any events in the events field applied to it. It is illegal to
    // send a snapshot without sending any events.
    google.protobuf.Any snapshot = 5;
}

// A reply message type for the gRPC call.
message CrudReplyOut {
    oneof message {
        CrudReply reply = 1;
        Failure failure = 2;
    }
}

// The CRUD Entity service
// Provides read and write operations for managing the entity state.
// For a CRUD entity an event represents also the whole state of the entity.
// A read operation only transport the state from the entity without changing it.
// Typical read operations are fetch and fetchAll.
// A typical write operation might transform the state of the entity and crate a new one.
// Typical write operations are create, save, and delete.
// Each write operation may generate zero or one event which is then sent to the entity.
// The entity is expected to apply these event to its state.
service Crud {

    // Create a sub entity.
    rpc create(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of a sub entity.
    rpc fetch(CrudCommand) returns (CrudReplyOut) {}

    // Save a updated sub entity.
    rpc save(CrudCommand) returns (CrudReplyOut) {}

    // Delete a sub entity.
    rpc delete(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of the whole entity.
    rpc fetchAll(CrudCommand) returns (CrudReplyOut) {}
}

I am not sure if it is better to define the types as an enum rather than a specific type of message. Usually enums are problematic in the evolution of proto files since any change to enum will break the parsers of older clients.

[ralphlaude]

@sleipnir thanks for the hints. i will change that to use dedicated type

[viktorklang]

@ralphlaude

I'm not sure I understand the following:

// The CRUD Entity service
// Provides read and write operations for managing the entity state.
// For a CRUD entity an event represents also the whole state of the entity.
// A read operation only transport the state from the entity without changing it.
// Typical read operations are fetch and fetchAll.
// A typical write operation might transform the state of the entity and crate a new one.
// Typical write operations are create, save, and delete.
// Each write operation may generate zero or one event which is then sent to the entity.
// The entity is expected to apply these event to its state.
service Crud {

    // Create a sub entity.
    rpc create(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of a sub entity.
    rpc fetch(CrudCommand) returns (CrudReplyOut) {}

    // Save a updated sub entity.
    rpc save(CrudCommand) returns (CrudReplyOut) {}

    // Delete a sub entity.
    rpc delete(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of the whole entity.
    rpc fetchAll(CrudCommand) returns (CrudReplyOut) {}
}

For instance, how would you implement the ShoppingCart example using this CRUD protocol?

[ralphlaude]

@viktorklang,
there are different operations (create/fetch/save/delete) for the protocol.
Each of those operations has a type and this type is used for mapping the CrudCommand for the called protocol operation. The CrudCommand contains the state of the entity to act on.
For a read operation, like fetch, the state can be transformed to another representation for the caller but no event should be emitted in this case. For this reason I stated read operations should no modify the state.
This can be applied to the ShoppingCart as follow:

• The AddItem will match save the protocol operation because the AddLineItem has the CommandType SAVE. AddItem as write operation could emit an event or failed.
• The RemoveItem will match delete the protocol operation because the RemoveLineItem has the CommandType DELETE. RemoveItem` as write operation could also emit an event or failed.
• The GetCart will match fetch the protocol operation because the GetShoppingCart has the CommandType FETCH. GetCart` as read operation should not emit an event or failed.

The ShoppingCart don't have a create operation. All ShoppingCart operations are mapped to CrudCommand containing the entity state. This state is then propagated through the SnapshotHander.
See:

• CRUD ShoppingCart proto - https://github.com/cloudstateio/cloudstate/pull/220/files#diff-e769f79128117c879a01d079e3ace0be
• CRUD ShoppingCart implementation - https://github.com/cloudstateio/cloudstate/pull/220/files#diff-5aa8af5771edf7800c507fe01a551248

The protocol don't need the fetchAlloperation at all, it will be removed.
I hope i could answer your question.

[viktorklang]

@ralphlaude TBH I don't think a service's external interface should need to disclose what type of state model it is using. For instance have a look at the CRDT example: https://github.com/cloudstateio/cloudstate/blob/master/protocols/example/crdts/crdt-example.proto Nothing in there tightly links the external interface to CRDTs. /cc @jroper @pvlugter @sleipnir @marcellanz Wdyt?

[ralphlaude]

@viktorklang,
it is true reagrding the service interface should not disclose the type model it uses.
The issue here is how the proxy should know what CRUD operation is beeing executed and right now i don't have an good answer to that.
The more general question is how the proxy should route a request to the right to CRUD operation, based on which criteria.
I would be happy to know more about possible options or other thoughts.
@jroper @pvlugter @sleipnir @marcellanz some thoughts?

[viktorklang]

@ralphlaude Would it make sense to have save / delete / create on the Context when handling the requests?

[sleipnir]

@ralphlaude TBH I don't think a service's external interface should need to disclose what type of state model it is using. For instance have a look at the CRDT example: https://github.com/cloudstateio/cloudstate/blob/master/protocols/example/crdts/crdt-example.proto Nothing in there tightly links the external interface to CRDTs. /cc @jroper @pvlugter @sleipnir @marcellanz Wdyt?

I agree. Just remember that the user can link directly to what http operations he expects when annotating the proto file with http resources.

@ralphlaude Would it make sense to have save / delete / create on the Context when handling the requests?

I think it makes more sense to map this in the method annotation. @ralphlaude what annotations are available to the user for CrudEntity? I think you can parameterize the type of invocation (get, save, delete, anything) as an annotation parameter, I think this is more fluent as it is similar to what developers are already used to when using web apis like spring mvc or jax -rs

[sleipnir]

Especially since the main difference between an EventSourced and CRDT is the annotations used in the entities, to change this would be to break the symmetry of the APIs.

[jroper]

I think the direction being taken here isn't quite right. This is back to front:

service Crud {

    // Create a sub entity.
    rpc create(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of a sub entity.
    rpc fetch(CrudCommand) returns (CrudReplyOut) {}

    // Save a updated sub entity.
    rpc save(CrudCommand) returns (CrudReplyOut) {}

    // Delete a sub entity.
    rpc delete(CrudCommand) returns (CrudReplyOut) {}

    // Fetch the state of the whole entity.
    rpc fetchAll(CrudCommand) returns (CrudReplyOut) {}
}

If the user function implements that, then we're saying the user function is the store, responsible for managing state. That's opposite of what Cloudstate is intending to achieve. If the proxy implements that, and then the user function invokes those calls back to the proxy, well that's closer to what Cloudstate is attempting to achieve, but it's also not the Cloudstate way of doing things - the user function isn't supposed to concern itself with retrieving state, that's meant to be the domain of the proxy.

Here's what the protocol that I envision would look like:

service Crud {
    // A streamed connection for handling commands to a particular entity.
    rpc handle(stream CrudStreamIn) returns (stream CrudStreamOut) {}
}

message CrudStreamIn {
  oneof message {
    CrudInit init = 1;
    Command command = 2;
  }
}

// When a crud entity is first activated, it will receive this message first
// before any commands are sent, this will tell it the service it's for and
// the id of the entity it's for, and if the value for the entity already
// exists, it will contain that as well.
message CrudInit {
  // The name of the service that implements this CRUD entity
  string service_name = 1;
  // The id of the entity
  string entity_id = 2;
  // The value of the entity, if the entity has already been
  // created.
  google.protobuf.Any value = 3;
}

message CrudStreamOut {
  oneof message {
    CrudReply reply = 1;
    Failure failure = 2;
  }
}

message CrudReply {
  // The command being replied to
  int64 command_id = 1;
  // The action to take for the client response
  ClientAction client_action = 2;
  // The action to take on the crud entity
  CrudAction crud_action = 3;
}

message CrudAction {
  oneof action {
    CrudUpdate update = 1;
    CrudDelete delete = 2;
  }
}

// Update the persisted value of the crud entity.
// If the entity is not yet persisted, it will be created.
message CrudUpdate {
    // The value to set.
    google.protobuf.Any value = 1;
}

// Delete the persisted value of the crud entity
message CrudDelete {}

This follows the same pattern as for event sourced and CRDT entities. Each CRUD entity is a single value that gets sharded across the Akka cluster, and when active, is stored in memory by the user function. When a command is received for a particular entity, if there's no active gRPC handle stream for that entity, a new stream is started, the value for the entity is looked up from the database, and then an init message is sent to the user function containing the value (or no value if not present in the db). Then the command is sent, and the user function can reply, optionally sending CRUD action, which can either update or delete the value from the database. After a period of inactivity, the entity will be shut down, just like for event sourced entities.

[ralphlaude]

@jroper your proposal for the Crud protocol is very clean 👍.

[blublinsky]

James's design look nice, but begs the question of how do we really position Cloudstate. So far I was assuming that it is mostly a stateful scalable backend for serverless.
If this is the case, then this proposal is too clever for average developer, who things in terms of CRUD
If not then we need to position Cloudstate slightly differently

[sleipnir]

@blublinsky Is that in fact what James is explaining is how he imagines or flow of events occurring and not an API for the end user. In the end user API, it is well built that you will notice a similarity as the usual CRUD model.

[sleipnir]

@ralphlaude I believe that the same flow described by James for CRUD should be taken into account for KV support. ;)

[ralphlaude]

@sleipnir i was thinhking about the protocol for the Key-Value support and it is pretty similar to the CRUD protocol described by @jroper. I didn't elaborate more on that now but it seems to me it could work that way.

[ralphlaude]

@blublinsky you can take a look here (#220) and every insights are welcome

[pvlugter]

Will close this issue now. The initial PR in #220 is merged, and there are follow-up issues created:

• Add instrumentation for value-based entities (Entity) Add instrumentation for value-based entities (Entity) #474
• Add Cassandra native support for value-based entities (Entity) Add Cassandra native support for value-based entities (Entity) #475
• Add Spanner native support for value-based entities (Entity) Add Spanner native support for value-based entities (Entity) #476
• Discussion: pluggable storage support for value-based entities (Entity) Discussion: pluggable storage support for value-based entities (Entity) #477
• Add docs for value-based entities (Entity) Add docs for value-based entities (Entity) #478