Questions.

[juliolitwin]

Yo ~

I'm following TofuECS and it looks quite promising. How is the status of the project currently?

• Apparently you're refactoring, so it's not ready for production yet, right?
• What are the goals you have in mind with TofuECS (future)?
• Can TofuECS compare with other frameworks (performance, optimization, API)? It would be amazing to have a benchmark, there is a DefaultECS benchmark compared to the others, in case you want to compare. :)

Anyway, the project is getting amazing.
Cheers!

[njelly]

Hey that's cool to get a notification. :)

EDIT: The issues I was talking about here have been addressed, you can probably use this for production. Up to you, the reader.

1. No, this is definitely not ready for production. The original implementation is working fine with tests, but when trying out some practical use cases in Unity, I'm getting crashes. I do have a neat Conways Game of Life sim that I've been using to benchmark everything, although it has been removed from this repo for now. See before commit a8582759634757145b22c5d50522ce7cc213b333. I'm doing some refactors currently to hide away a lot of the unmanaged code and fix crashes but I wasn't expecting anyone to notice :)

2. The goal is to increase my own knowledge and also to eventually have a production-ready ECS solution that can be used in Unity (or with Monogame, etc.). Eventually I'd like to have it working with third-party open source network solution, roll back engine, and physics engine (deterministic, no lock step).

3. I haven't even started to compare this to other frameworks. I've been benchmarking locally for now to make sure I don't fall backward with new features or refactors. I don't have any particularly revolutionary ideas about how to structure an ECS so I'm not trying to beat out anyone else, but definitely it ought to be comparable at the very least.

There's lots of other ECS solutions (Entitas, LeoECS, entt, etc.) with a lot more people contributing, but maybe this project can be even more light weight and easier to understand?

Thanks for following. I'll be pushing commits frequently outside of my day job haha

[juliolitwin]

It's so cool!

Is it amazing how the space for 'rollback-networking' is growing lately, perhaps because of Quantum? I don't know, but I'm happy to see some frameworks in development! I had seen about rollback within the code, not expecting that its end goal would actually be for rollback-networking.

And, amazingly, you're working with unmanaged code, it's a difficult sea to explore, but when it's working, it's just happiness. :P

Do you intend to re-add an example project using TofuECS?

In case you are curious about a rollback-networking solution project under development:
PleaseResync

Do you know another open-source third-party regarding the rollback solution? Without being the classic GGPO.

I'll be following the project, and possibly want to test it soon.
And thanks for making this project open source.

[njelly]

Yes, Quantum is what I've used professionally. It's been pretty cool because it has all of the physics, rollback, and ecs solutions packed into one, but I'm not a fan of the DSL or code gen and the separate project. Other repos seem to use code gen too and maybe it is necessary, but I'd really love to just drop a dll into Unity and not have to maintain a plugin.

Here's the example Unity project I'm working on as a separate repo, not currently functional. I imagine there are some generic helper classes for getting data into the sim when working in Unity, I had those here earlier as well but since I want this project to be Unity-agnostic that will be separate.

GGPO is the rollback library I'm aware of, as well as its Unity port. I was hoping I could use that but I hadn't heard of the one you linked.

[juliolitwin]

Hey @njelly

Since you are working with unmanaged code, would there be any advantage to you using unsafe collections? Trying to get a new performance gain with collections of a lower level.

Examples:
Unsafe Collectons
Unsafe Collections (fork fhlom)

[njelly]

Hey @njelly

Since you are working with unmanaged code, would there be any advantage to you using unsafe collections? Trying to get a new performance gain with collections of a lower level.

Examples: Unsafe Collectons Unsafe Collections (fork fhlom)

Interesting. I'd want to do a performance comparison between just the regular TComponent[] arrays being used now and this library first.

[juliolitwin]

Okay, I'm looking forward to seeing the benchmark.

Is there any danger of TofuECS having some (architectural) refactoring? Or can I start a project?

A question about the checksum, would there be any way for Tofu to provide the checksum for comparison or would it be better provided by the networking solution?

Another question about the Garbage Collection on Tofu, is there any GC or totally zero at runtime?

Cheers.

[njelly]

There's probably some level of "danger" but for a hobby project I wouldn't worry too much. I'm hoping to avoid any further API changes and any major changes would have to have a very good reason at this point. I haven't been working on this framework for very long (a few months at this point) so it isn't battle-tested. But for the example I have so far and the tests I've written, it runs very well.

I would probably leave it to the networking solution to provide the checksum. There's no reason this ECS shouldn't be deterministic with the same inputs so long as you don't use float values in your calculations. You could also maybe roll out your own checksum solution by checking the state using the GetState<TComponent> API, although I haven't tried that. Maybe if there is enough demand that could be a feature I can look into.

The delegates created in functions like GetAndModify, ModifyUnsafe etc. on the component buffer do create garbage. There's ways to work around that, and I'll provide examples, but here's a good blog post explaining the issue.. I'm not entirely sure what the consequences are for huge, complex projects, but it is something I'll keep in mind as I work on this framework.

[njelly]

@juliolitwin

Wow. The unsafe array library really helps. Checkout the unsafe-array branch for TofuECS. For my COGL clone I've been using to benchmark, I've gotten down to 60 bytes of gc alloc per tick, with 0 bytes coming from the ECS itself (although I'm caching a bunch, literally never creating any new variables outside of the Initialize() method for BoardSystem). There's a couple other places in the framework I could remove gc alloc (when creating iterators on buffers) and maybe I see if I can get those cached as well.

The changes on that branch completely remove the delegates, you're now able to access pointers to the buffer directly. It feels very clean, imo. It's pretty different from the syntax on the main branch.

Again, wow. That's a very cool repo you linked. Thanks :)

[juliolitwin]

Oh, this is amazing @njelly ! Congratulations. :)

I tested the COGL and saw that they are running smoothly, with the 60 bytes with GC alloc. I didn't investigate, but you said it's just the BoardSystem that's allocating, right?

Looking at the TofuECS code, many things could be improved, but the level of complexity can increase a lot. As you said, some things need to be cached (pools everywhere :P).

I saw that you still use some collections from .NET itself, did you get to change and test? Because I believe there may be a performance improvement, I don't know.

There's also a library of collections with pools already defined, it's a great library too, so you'll already have the pools by default.

Repository:
Collections.Pooled

In the COGL project, you are using compiled DLL's, probably you compiled it without the #def UNITY, correct? This define is great because it uses the native Unity library to use their own memory allocation, so a disadvantage of using the dlls in Unity instead of the project, would be the lack of these defs for compilation.

In this case, instead of using the 'Buffer' (alloc/copy) of .NET, what do you think of using native libraries (memcpy) or for example Rpmalloc?

Repository:
Rpmalloc-CSharp

Have you already tested how COGL behaves with rollback? How is the performance?

I believe I talk too much and travel too much with my ideas, hahaha!
Cheers.

[njelly]

@juliolitwin yeah BoardSystem just allocates once on initialize (I probably over did it, but w/e), but no extra memory is allocated while the sim is running. Those 60 bytes I think come from the gc itself (?) and a formatted string on the Unity side.

The pools are interesting. I think I was able to solve the Iterator problem by just using Next / NextUnsafe method with a user provided ref int for incrementing. Should be fine but you just have to make your int before your while loop, which may be annoying.

I'll keep those other repos in mind. I think the most important thing at this point is to actually use TofuECS in a real project (TofuECS_Rogue is what I'm working on) and fix bugs/optimize as needed rather than try to over-optimize at the start.

[Leopotam]

@njelly

There's lots of other ECS solutions (Entitas, LeoECS, entt, etc.) with a lot more people contributing, but maybe this project can be even more light weight and easier to understand?

You are wrong, leoecs/leoecs-lite and almost all extensions were created and maintained by one-man team :) All community fixes are about mistypes at README and so on. Example projects - yes, but amount of them low enough.

About lightweight and easier to understand - we can count LoC in both repos to detect which one is lighter/easier :)

Main goals of leoecs:

• no unsafe.
• pure C# implementation with possibility of using any managed types on user side.
• no codegen.
• to be faster than paid (in 2018) Entitas.

Main goals of ecslite:

• same as goals of leoecs.
• performance fixes on random access to any component on any entity without additional work on user side.
• extensibility - small core with public api + extensions around it.

[njelly]

@Leopotam oh hey, love your repo!

Ah I wasn't aware, I just knew yours was very popular. Very nice work! I think my point was moreso that repos like yours are more established, have been developed for longer (more time for feedback, if not actual PRs) and I've only been working on this one for a couple months (and so there's room for this one to become something different). Thanks for checking my little project out! :)