Support for layers

[jshph]

Hi Peter, great project -- I happened to stumble across this on YouTube and it's exactly the kind of lightweight solution I'm looking for.

I'm curious if you have considered support for adding layers of images? My use case would be to gradually add and remove certain components from the Excalidraw diagram over the course of a presentation. My opinion is that the easiest way to do this as a user would be to:

• transition between different images (the Excalidraw drawing) of the same size, in the background, with everything else being the same.
• keyframe between the layers (either completely, or with transparent backgrounds so both can be seen)

Let me know if that brief description makes sense; happy to elaborate or perhaps contribute on any fronts that make sense for you.

[peter-kuhmann]

Hi Joshua,
Thanks for your kind words! I apologize for answering this late – I was on a 4-week long vacation and had not a suitable device to write this long answer. 😅

---

Regarding your question: Yes, I considered such a feature. Let me give you a short insight into how it started:

• I wanted to present Excalidraw scenes progressively.
• I tried out this tool: https://github.com/dai-shi/excalidraw-animate – Blocking problems: The order of the animation steps can not be changed.
• Used Apple Keynote to present my Excalidraws with panning and zooming.
• I built Pan'n'Zoom mimicking the Keynote effect – just more straightforward.

As you can see, I stuck to that concept. Panning and zooming are necessary; "revealing" elements can be done by increasing the next keyframe's size – that was the idea. But it's suboptimal.

Having both features (step-by-step panning/zooming and step-by-step masks) would allow to:

• reveal elements one by one in a smaller area without the need for any movement (reduced cognitive load),
• move to different areas when one wants to jump to the "next big picture."

My initial "MVP" list contained the feature "allow the user to blur non-keyframe area" – but I ditched it. I think it would be awesome to reveal elements/areas step-by-step!

---

While hiking in Slovenia, I had time to think about the problem. I see two options from the architecture perspective:

• (a) Different images (as you suggest) for the keyframes (image A for keyframes 1-4, image B for keyframes 5-12, ...).
• (b) Mask editor within the tool: Let the user define which parts of the image are visible for each keyframe.

Intuitively, I would pick (b) because of the following aspects:

• Storing and exporting multiple images (method (a)) would come at the high cost of big files and long loading times.
  • Storing per-keyframe mask information (SVG?) is probably way cheaper.
  • Displaying mask SVGs is probably way faster.
• Managing many image variants can become frustrating.
  • I also developed the tool to be able to replace just one/the image when I fix a typo in the image ...
  • ... with many images, this would become a tedious task.
• Let's you animate "area-revealing" with images you don't have the source file for anymore to create variants for it ... respectively: The user does not need to use an image editor to create variants.
• Masks could be changed/adjusted super fast in the editor.

When I had this thought of a "mask editor", immediately the Adobe Lightroom mask drawing mode (red mask -> https://helpx.adobe.com/content/dam/help/en/lightroom-classic/help/whats-new/2022/select_subject_masking-new.png.img.png) popped up in my head. 😅

---

I am super interested in your thoughts! I am also happy to brainstorm together! 🙂

Peter

[jshph]

Agreed on all fronts, and thanks for the additional context on the MVP. Hope you had a good time off :)

The SVG mask idea makes sense based on how I would use it -- perhaps an optional, editable mask that can be applied or copied to each keyframe. A simple version of that could allow for drawing a rectangle over the image, and reduce the opacity / gray-out the areas outside of the rectangle. The rectangle could be stored as SVG or simply as a bounding box, rendered for each keyframe.

Similar to Lightroom would be the layer masks in Adobe Illustrator/Photoshop, where the shapes used for the mask can be vector-based.

[peter-kuhmann]

It might make sense to restructure the app/editor to have a more complex but still simple timeline view – similar to a video editor.

The image below shows a brain dump:

• Timeline view at the bottom.
• Each keyframe can have a position/bounding rect and/or mask.
• If one is not specified, the previous position/bounding rect and/or mask is used.
• If the first keyframe does not have a position -> full bounding rect is used.
• If the first keyframe does not have a mask -> mask revealing everything is used.

What do you think?

[jshph]

Yeah that all sounds good to me. It lines up with my mental model of how Adobe's masks work -- initialized to the full view, with the difference being that if there is a prior keyframe then that keyframe's mask gets inherited.

[peter-kuhmann]

Hey @jshph!

This week, I will experiment with run-length encoded mask "bitmaps." It's probably way simpler to encode a "mask canvas" to a 0/1 run-length encoded bitmap. I will probably use that approach if it doesn't consume too much memory space.

[jshph]

That would be a clever bit of engineering. Thanks Peter! Eager to hear where it goes.

[peter-kuhmann]

I create some simple tests regarding RLE encoded bitmaps.

Assuming

• we present an image with a resolution of 2000x3000 pixels
• and use a mask with a resolution of 500x750 (so 1/4, which should be precise enough)
  the RLE-encoded bitmap mask requires (depending on the actual drawings) ~5-15KB, which is acceptable.

The format is "{width}x{height}:T{runLength}M{runLength}T{runLength}M...".

---

I tried to (mis)use UTF-8 to store run lengths using fewer bytes (via String.fromCharCode(runLength)) to compress the RLE-encoded bitmap even more.

Format is "{String.fromCharCode(width)}{String.fromCharCode(height)}{firstRunMasked ? 'M' : 'T'}{String.fromCharCode(runLength)}{String.fromCharCode(runLength)}..."

Results: UTF-8-hacky-RLE-encoded bitmaps are 3x smaller on average. 🤩 So around 1,5 - 5KB for the bitmap resolution state above. That is very acceptable.

---

To me, this looks like the best option:

• It enables hand drawing.
• Memory consumption is acceptable.
• Easy to use. No complex SVG conversions are required.

Peter

[jshph]

Hi Peter, sorry I've had a pretty busy weekend / week!

That's incredibly promising, thank you for taking the time to share about it!! I'm not sure I see any downsides from the UTF-8 optimization; potentially, char codes are less interpretable, but not sure that is important to optimize. Certainly, a lightweight experience would be much appreciated :)

Let me know how I could help here -- any PRs to review?

Josh