Use a memory limited hashset with `LocalVocab`#1570

Open

s1dharth-s wants to merge 16 commits intoad-freiburg:masterfrom s1dharth-s:Localvocab-with-Limit

+470-4

Commits on Nov 11, 2024

Initial changes to use memory limited hashset with LocalVocab
s1dharth-s
committed
Change template order for hash set. SizeGetter now returns ad_utility::MemorySize
s1dharth-s
committed
Use type alias for long type signature of absl::node_hash_set<...>
s1dharth-s
committed
add contains
s1dharth-s
committed
Add comments and seperators between trivial functions for better readability
s1dharth-s
committed
Pass AllocationMemoryLeftThreadSafe object to LocalVocab to manage the memory usage instead of ad_utility::MemorySize
s1dharth-s
committed
Add destructor for CustomHashSetWithMemoryLimit which frees up memory used by its AllocationMemoryLeftThreadSafe object.
s1dharth-s
committed
Include sizeof(std::string) to the dynamic memory calculation of LiteralOrIri objects. Also added comments.
s1dharth-s
committed
Take into account the slot size for the hash set. Refactor logic to increase and decrease memory usage
s1dharth-s
committed
add tests
s1dharth-s
committed
move \´slotMemoryCost inside the class\´
s1dharth-s
committed
Rename CustomHashSetWithMemoryLimit to NodeHashSetWithMemoryLimit. Change the default template argument of SizeGetter from SizeOfSizeGetter to DefaultValueSizeGetter.
s1dharth-s
committed
Change slotMemoryCost from size_t to MemorySize. Changed all variables used to count slot size to have same name. Initialize memoryUsed_ and currentNumSlots_ in class.
s1dharth-s
committed
Update SizeGetters to only account for dynamic memory usage.
s1dharth-s
committed
Set default memory limit of LocalVocab unlimited. Remove unwanted SizeGetter variables.
s1dharth-s
committed
Refactor tests.
s1dharth-s
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use a memory limited hashset with `LocalVocab`#1570

Commits on Nov 11, 2024

Initial changes to use memory limited hashset with LocalVocab

Change template order for hash set. SizeGetter now returns ad_utility::MemorySize

Use type alias for long type signature of `absl::node_hash_set<...>`

add `contains`

Add comments and seperators between trivial functions for better readability

Pass AllocationMemoryLeftThreadSafe object to `LocalVocab` to manage the memory usage instead of `ad_utility::MemorySize`

Add destructor for `CustomHashSetWithMemoryLimit` which frees up memory used by its `AllocationMemoryLeftThreadSafe` object.

Include `sizeof(std::string)` to the dynamic memory calculation of `LiteralOrIri` objects. Also added comments.

Take into account the slot size for the hash set. Refactor logic to increase and decrease memory usage

add tests

move \´slotMemoryCost inside the class\´

Rename `CustomHashSetWithMemoryLimit` to `NodeHashSetWithMemoryLimit`. Change the default template argument of `SizeGetter` from `SizeOfSizeGetter` to `DefaultValueSizeGetter`.

Change slotMemoryCost from `size_t` to `MemorySize`. Changed all variables used to count slot size to have same name. Initialize `memoryUsed_` and `currentNumSlots_` in class.

Update `SizeGetter`s to only account for dynamic memory usage.

Set default memory limit of `LocalVocab` unlimited. Remove unwanted SizeGetter variables.

Refactor tests.