Main+Store: NeDB persistence and integrity + dependency update

[Svallinn]

Pull Request Type

• Bugfix
• Other

Related issue

I'm sure there are issues related to this, but I wouldn't close them until some time has passed in order to check if this passes the test of time (due to the integrity problem being such a rare heisenbug and dependent on timing).

Description

This PR replaces the nedb-promises dependency with @seald-io/nedb (which the former used internally).
Given that the internal dependency now supports Promises natively, nedb-promises is now fundamentally obsolete.

Additionally, clean up code in the main process has been freshen up and data store compaction is performed before terminating the main process to ensure that all former operations in the queue have finished executing.
This will hopefully prevent any integrity issues related to the datafiles in the case of standard process terminations.

Testing

Everything seems to be working correctly after the dependency switch and appropriate refactoring.

Desktop

• FreeTube version: development (8025945)

[PikachuEXE]

Add some comments?

[Svallinn]

Add some comments?

@PikachuEXE
IMO, anyone with IntelliSense or an internet connection can figure out what it does.
Comments like those should be for more obtuse developer-made code, not for standard built-in objects' methods.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[efb4f5ff-1298-471a-8973-3d47447115dc]

Leaving this for @absidue

[PikachuEXE]

On MacOS (dev) when I quit the app it looks stuck for a long time (>10s)
Not sure how to display that it's doing stuff instead of stuck in something...

Edit: to reproduce, import subscriptions/clear subscriptions/playlist/history with large enough number (I did it with playlists on my playlist branch)

[Svallinn]

On MacOS (dev) when I quit the app it looks stuck for a long time (>10s) Not sure how to display that it's doing stuff instead of stuck in something...

Edit: to reproduce, import subscriptions/clear subscriptions/playlist/history with large enough number (I did it with playlists on my playlist branch)

What's your basis on it appearing stuck? Genuinely asking since I don't know the MacOS details.
You couldn't have understood that from a window closing, since the compactation occurs after the windows are gone, that's why I'm asking.

Regardless, that also implies that, if you take those actions without my PR, these two things were probably happening before (depending on how activate event functions, these may be slightly off):

• The datastores were never being compacted after the app ran the first time.
  My guess is that, when you kill all the windows, it still leaves the original datastore handlers in memory and just reuses them when you spawn a new window (on that same system boot obviously, rebooting gets rid of those handlers and it'll just compact on datastore load when opening the app on the next boot).
• The prior point implies that there was probably a "delay" to process that large number of lines before as well, but it was only when there was no handlers to be reused and had to load new ones.
  Naturally, this would only happen if it's the very first time you open the app on that particular system boot, or one can probably force this somehow by killing Electron's main process for good and reopening the app.

These are, at best, speculations since I can't test, feel free to verify and let me know.
Also, "system boot" is probably not the right term for what I'm trying to transmit, but I think you can understand what I mean.

[PikachuEXE]

On MacOS one can quit an app by right clicking the icon on the dock

And with some conditions (probably when there are many changes to compact) the app would appear unresponsive (but it's actually compacting without any visual response)
So if user right clicks again it looks like this

I know FT is doing stuff but I guess most users don't know that
Just thinking if there is an easy way to NOT make FT look stuck and make user more likely to click on Force Quit

[Svallinn]

It's a tough situation to work around...
I still gotta say, 10 seconds feels like a fucking eternity even if you have 250 subscriptions removed or so.
Is NeDB that slow? Probably, but...
I hate to push this onto you, but can you remove these two from cleanUpResources in the main.js

session.defaultSession.clearCache(),
session.defaultSession.clearStorageData({
  storages: [
    'appcache',
    'cookies',
    'filesystem',
    'indexdb',
    'shadercache',
    'websql',
    'serviceworkers',
    'cachestorage'
  ]

build and check if that still happens?
Just wanna make sure it's NeDB before I even consider putting in the effort of thinking about this.

[PikachuEXE]

Let me see how to reproduce that long exit delay consistently first
But I never experiences this until 1-3 weeks ago
This PR is the only one that I can think of (as it alters exit logic)