Maybe and empty list/set to DynamoDB

[pjrt]

At the moment, it is possible to pass an empty list or set when encoding items. Of course empty sets, lists and strings aren't allowed in DynamoDB and such requests will fail.

Currently we are getting around this by using a specialized version of item and attr that work on Maybes; empty lists, sets and strings are converted to Maybes and Nothing is simply removed from the list (since Dynamo is fine with missing fields).

I noticed that a DNull case was added to DValue. Nulls can't be passed as values to Dynamo but maybe another field called DEmpty? Then use this field for Nothing and empty lists and sets. Then we could pattern match on it and remove it out of the final request.

[pjrt]

So I see we are aware of this already.

Is there a reason to not do this?

[aristidb]

CC @ozataman

[pjrt]

So this is interesting. Apparently you CAN put null values into items. Though I'm not 100% sure what this means. The docs explicitly state that you can't put Null values:

When you add an item, the primary key attribute(s) are the only required attributes.
Attribute values cannot be null.

yet...

streamId and time make up the key.
Though now that I think about it, maybe that sentence in the AWS doc meant to say:

When you add an item, the primary key attribute(s) are the only required attributes and
it (they) cannot be null. Non-PK attributes can be null.

...since you can't make the hash-key or the range-key null (according to my test).

I would guess this means we CAN use null for empties and Maybe?

[pjrt]

Ok, so I tried this and it CAN be done, but I think it makes the interface kind of confusing.

Essentially, people would see "Null: True" or simply a missing field when they enter an empty value or a Maybe. This is can be confusing since you would expect something to show up in Dynamo, yet nothing shows up or the confusing "Null: true" shows up. Additionally, the behavior of getAttr and getAttr' also become confusing (is Null: True a Maybe Int or a Set Int? If the value is missing, does getAttr = getAttr'?).

I think the best option would be to have another attr function (called attr', similar to getAttr') that takes a DynVal a => Maybe a and places a DEmpty for Nothing into the Attribute list. This value would then be filtered out of the final item. This is cleaner since then users would use getAttr' to get that same value out.

Additionally, we could add some types: NonEmptyText and NonEmptySet a and instances for them. Then make the user guarantee their non-emptiness.

[tmbull]

Have you guys given any more thought to Maybe over the past few months? I have a scenario where I would like to parse a non-key field as Maybe Bool, but it does not seem like there is a clean way to do this currently.