Benchmarks

Description of the system

For us to run the experiments we will use a machine proprietary of the Web Semantic Research Group (WESO) of the University of Oviedo. The machine has the following characteristics:

• CPU: Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (12 cores and 24 threads)
• RAM: 40GB
• OS: Ubuntu 20.04.3 LTS

Description of the dataset

We have created the 1-University LUBM dataset, with the following characteristics:

• Subjects: 17,175
• Predicates: 18
• Objects: 13,948
• Triples: 151,388
• Size: 8.268 MB

RemoteHDT Import (N-Triples to ZARR)

⚠️ Note that the following is just a comparison between the different released versions, and should not be seen as the actual performance of the program in your system.

Version	Description	Threads	Time (s.)	RAM (GB)	Resulting (MB)	Complexity	Compression
v0.0.1	Baseline	1				O(n3)	❌
v0.0.2	Multi-threaded and cartesian product implemented	16				O(n3)	❌
v0.0.3	Loop simplified	1				O(n)	❌
v0.0.4	Compression implemented	1				O(n)	✅
v0.0.5	Multi-threaded, loop simplified and Compression	16	156.12	11.1	4.443558	O(n)	✅
v0.0.6	v0.0.5 + using Chunks for creating the dataset	16	14.08	0.80	204.4581	O(n)	✅
v0.0.6	Getting back to a pseudo multithreaded version	16	33.00	0.03	204.4581	O(n)	✅

RemoteHDT Deserialization

Version	Description	Threads	Time (s.)	RAM (GB)	Compression
v0.0.7	Baseline	1	28.57	3.95	✅

RemoteHDT vs. SPARQL Queries

⚠️ Note that the following tests were performed using the SPO orientation for getting the first subject

Version	Description	Threads	Time (s.)	RAM (GB)	Compression
v0.0.7	Baseline	1	90.94	12.33	✅
v0.0.8	Chunk-based approach	1	2.81*	0.01	✅
v0.0.9	Sparse matrix-based approach				✅

We have performed the following query over the 1-LUBM dataset:

PREFIX :  <http://www.Department14.University0.edu/>
SELECT *
WHERE {
    :GraduateStudent94 ?predicate ?object .
}

PREFIX :  <http://www.Department14.University0.edu/>
SELECT *
WHERE {
    ?subject ?predicate :GraduteCourse39 .
}

The results are in seconds:

											Average (s)	STDEV (s)
RemoteHDT (s)	0.00101	0.0008213	0.0007282	0.00138	0.00107	0.0007605	0.0006927	0.0006927	0.00104	0.0006753	0.0009	0.0002
SPARQL (s)	0.0510	0.0680	0.0120	0.0110	0.0140	0.0110	0.0120	0.0170	0.0100	0.0120	0.0218	0.0204