Local store

[gedw99]

Dynamo dB is cool but I was wondering if you are tempted to reconsider also supporting self stores too. So people can use AWS or local

genji is similar to dynamo dB in that it’s badger dB ( a kv store of sorts ) with a sql document dB on top.

been using if for a large project and it’s been ok. A few shaky edges but easy to work around until it’s better.
Speed is really great.

https://github.com/genjidb/genji

there is no HA aspects integrated however many have managed to use raft with badger dB for replication.

I am hoping this is of interest and if you want some help with it because the basis of event horizon looks in line with how I think CQRS is best used.

[joonas-fi]

The previous version of EventHorizon worked with local-only store (BoltDB), and I was meaning to add Raft on top. I quickly realized it's a daunting problem, and opted for DynamoDB instead.

Once I get EventHorizon into a satisfactory working condition, I plan on making the direct dependency on DynamoDB an interface, and thus one can make plugins for different storage engines.

I used BoltDB in the previous version, so that's somewhat likely to be added as plugin next to DynamoDB. But I am currently not planning to add a Raft layer - if one wants high-availability storage, then the HA should be provided by the storage engine itself (like DynamoDB does now in my current design)

[gedw99]

The previous version of EventHorizon worked with local-only store (BoltDB), and I was meaning to add Raft on top. I quickly realized it's a daunting problem, and opted for DynamoDB instead.

Once I get EventHorizon into a satisfactory working condition, I plan on making the direct dependency on DynamoDB an interface, and thus one can make plugins for different storage engines.

I used BoltDB in the previous version, so that's somewhat likely to be added as plugin next to DynamoDB. But I am currently not planning to add a Raft layer - if one wants high-availability storage, then the HA should be provided by the storage engine itself (like DynamoDB does now in my current design)

Yeah i dont want to use others clouds and their Databases.

Its possible i think. But yeah requires more work, etc

Most people are using this:
https://github.com/lni/dragonboat

Example
https://github.com/atomix/dragonboat-raft-storage-node

[joonas-fi]

Yeah i dont want to use others clouds and their Databases.

Out of curiosity - why? If the data is encrypted and because the cloud provider doesn't have access to the encryption keys, they can't access the data. You get a highly available datastore that you don't have to manage.

Managing one's own highly available cluster of databases is not free regarding work hours either.

[gedw99]

The project requirements in terms of security and policy. The EU is funding projects to get the EU companies off the US mega corps cloud and databases

On Tue 22. Dec 2020 at 12:25, joonas.fi ***@***.***> wrote: Yeah i dont want to use others clouds and their Databases. Out of curiosity - why? If the data is encrypted and because the cloud provider doesn't have access to the encryption keys, they can't access the data. You get a highly available datastore that you don't have to manage. Managing one's own highly available cluster of databases is not free regarding work hours either. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMVPLFEV72SLLBNNASU6ERLSWB633ANCNFSM4VBG2DTA> .

-- Mobile: +4915231894553 Telegram: http://t.me/gedw99

[joonas-fi]

I don't get it, if the data is encrypted and the encryption keys never touch AWS, the only threat from AWS is that they take your database down. They can't look at what's inside the database..

[gedw99]

I guess the only example that makes it sink home is if there was a war between the US and the EU. Mobile: +4915231894553 Telegram: http://t.me/gedw99

…

On Tue, 22 Dec 2020 at 13:12, joonas.fi ***@***.***> wrote: I don't get it, if the data is encrypted and the encryption keys never touch AWS, the only threat from AWS is that they take your database down. They can't look at what's inside the database.. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.