Why does StableBuffer<X> store [var X] instead of [var ?X]?

[darkdrag00nv2]

Hi, thanks for this library!

I have noticed a fundamental difference between StableBuffer and the Buffer from the base library. StableBuffer stores [var x] while Buffer stores [var ?X]. This is a small difference but it makes a few things tricky.

Example:
initPresized: As a user, I would expect the method to initialize the buffer of given size when I call it. Instead, this method only updates the capacity and the actual allocation of buffer happens on first add. That's okay but slightly different from expectations. I believe it had to be done this way because we cannot initialize the buffer with all null. So we needed a default value as placeholder which was only available during add. Any method which needs a placeholder will be difficult to write in StableBuffer.

I am sure there is a reason for this. Could you please explain here or even better document it somewhere? Thanks!

[ByronBecker]

Hey @darkdrag00nv2, thanks for the question!

The reason for this is that the Buffer module in Motoko used to be of type var X.
https://github.com/dfinity/motoko-base/blob/10ac7499cb3772716c6ef125d60561e84a835522/src/Buffer.mo

This change to var ?X was made in October of last year, and I really didn't have the time to make these updates once the change was live.

@tomijaga Was ahead of the curve when he submitted #4 (you can read the back and forth discussion there), and at the time I was concerned with breaking backwards-compatibility with the data stored by users of the original StableBuffer library.

At this point since it's been a considerable amount of time since the base library has changed, I would be for pulling in those changes, pushing a new major version v1.0.0, and switching the default branch people see to something like v2 (to avoid clashing with people who are just putting main for the version field of their vessel.dhall file (instead of using version tags).

What do you think about making the change from var X -> var ?X? I know it would retract some of the work you've done thus far, but if ICDevs will fund this work I'm happy to accept these changes.

The last thing I want to happen is for

[darkdrag00nv2]

Thank you for the detailed explanation!

What do you think about making the change from var X -> var ?X? I know it would retract some of the work you've done thus far, but if ICDevs will fund this work I'm happy to accept these changes.

Sure, I think moving to var ?X would be the right thing to do.

As per the forum thread, it'll be funded by ICDevs but we have to wait for the official approval. I would say we can merge my PRs in the current form and we can update them as well once we get to making the transition to var ?X.

[ByronBecker]

@darkdrag00nv2 Sorry for the back and forth, but I decided to pull the trigger and just migrate to using var ?X as the internal StableBuffer structure going forward.

Thanks to #4 from @tomijaga.

If you don't mind updating your PRs to adapt to the new [var ?X] internal element buffer, I'd be happy to merge them. This should also hopefully make any future PRs much easier in copying and pasting from the motoko-base implementation.

[darkdrag00nv2]

If you don't mind updating your PRs to adapt to the new [var ?X] internal element buffer, I'd be happy to merge them.

Sure, I'll update the PRs in a few days.