Skip to content

Latest commit

 

History

History
480 lines (453 loc) · 18.7 KB

README.md

File metadata and controls

480 lines (453 loc) · 18.7 KB

Evaluation

Table of contents

1. WebRTC IP address leakage evaluation
2. Performance impact evaluation of the containerised solution
References

1. WebRTC IP address leakage evaluation

Please read the section IP address leakage evaluation of the submission.

The leak evaluation was carried out on Linux Ubuntu 22.04 LTS, Microsoft Windows 11 23H2 and macOS Sonoma 14.5. Full details of the software and versions used in this evaluation are described in the table below:

Linux Windows macOS
Machine Dell Latitude 5520 (2022) Machine Dell Latitude 5540 (2023) Machine Apple MacBook Pro (13-inch, 2022)
CPU Intel Core i5-1145G7 @ 2.60 GHz CPU Intel Core i5-1335U @ 1.30 GHz CPU Apple M2
RAM 16 GiB RAM 8 GiB RAM 16 GiB
GPU Intel® Iris® Xe Graphics GPU Intel® Iris® Xe Graphics GPU Apple M2
Screen resolution 1920x1080 Screen resolution 1920x1080 Screen resolution 2560x1600
Host OS Ubuntu Desktop 22.04 LTS Host OS Windows 11 Pro 23H2 Host OS macOS Sonoma 14.5
Architecture x86-64 Architecture x86-64 Architecture arm64
Network configurations
Ethernet IPv4 address 203.0.113.20 Ethernet IPv4 address 203.0.113.20 Ethernet IPv4 address 203.0.113.20
Ethernet temporary IPv6 address 2001:db8::20 Ethernet temporary IPv6 address 2001:db8::20 Ethernet temporary IPv6 address 2001:db8::20
Ethernet permanent IPv6 address 2001:db8::21 Ethernet permanent IPv6 address 2001:db8::21 Ethernet permanent IPv6 address 2001:db8::21
Wi-Fi IPv4 address 203.0.113.30 Wi-Fi IPv4 address 203.0.113.30 Wi-Fi IPv4 address 203.0.113.30
Wi-Fi temporary IPv6 address 2001:db8::30 Wi-Fi temporary IPv6 address 2001:db8::30 Wi-Fi temporary IPv6 address 2001:db8::30
Wi-Fi permanent IPv6 address 2001:db8::31 Wi-Fi permanent IPv6 address 2001:db8::31 Wi-Fi permanent IPv6 address 2001:db8::31
Natively installed browsers on the host client
Mozilla Firefox v125.0.3 Mozilla Firefox v125.0.3 Mozilla Firefox v125.0.3
Google Chrome v126.0.6478.126 Google Chrome v126.0.6478.127 Google Chrome v126.0.6478.127
Microsoft Edge v126.0.2592.81 Microsoft Edge v126.0.2592.81 Microsoft Edge v126.0.2592.81
Opera v111.0.5168.55 Opera v111.0.5168.55 Opera v111.0.5168.55
Brave Browser v1.67.123 Brave Browser v1.67.123 Brave Browser v1.67.123
Safari v17.5
VPN and SOCKS clients
OpenVPN UDP client OpenVPN 2.5.9 OpenVPN UDP client OpenVPN GUI v11.48.0.0 OpenVPN UDP client Tunnelblick 4.0.1
WireGuard client v1.0.20210914 WireGuard client v0.5.3 WireGuard client v1.0.16
Mozilla Firefox built-in SOCKS client v125.0.3 Mozilla Firefox built-in SOCKS client v125.0.3 Mozilla Firefox built-in SOCKS client v125.0.3
Mozilla Firefox built-in HTTP/S client v125.0.3 Mozilla Firefox built-in HTTP/S client v125.0.3 Mozilla Firefox built-in HTTP/S client v125.0.3
Containerised Mozilla Firefox solution via docker
Docker image Ubuntu Server 22.04 LTS Docker image Ubuntu Server 22.04 LTS Docker image Ubuntu Server 22.04 LTS
Docker image architecture x86-64 Docker image architecture x86-64 Docker image architecture arm64
Containerised Firefox v125.0.3 Containerised Firefox v125.0.3 Containerised Firefox v125.0.3
Docker Engine v26.1.4 Docker Desktop v4.30.0 Docker Desktop v4.30.0
Docker Compose v2.27.1
Host Wayland compositor Mutter v42.9 Host Wayland compositor Weston (provided by WSLg) GitHub commit f227edd6 Host Wayland compositor N/A
Host X server (X.Org/XWayland) Host X server (XWayland) Host X server (XQuartz) v2.8.5
Host PulseAudio server PulseAudio (on PipeWire 0.3.48) Host PulseAudio server (provided by WSLg) GitHub commit 6f045ff0 Host PulseAudio server v17.0
WSL 2 v2.2.4.0
WSL 2 Linux Kernel with camera driver support v5.15.153.1 - Download
WSLg v1.0.61

The raw data are available in the following folder: preventing-webrtc-ip-address-leaks/3-raw-data/1-webrtc-leak-data.

2. Performance impact evaluation of the containerised solution

We are also evaluating our containerised solution in terms of performance and comparing it to a native, non-containerised solution.

Our containerised solution, initially designed for Linux, was later ported to macOS and Windows to demonstrate its viability on systems other than Linux. This is also why Docker was chosen, as it exists on these three operating systems.

The evaluation of the performance impact of our dockerised solution compared with running Firefox natively was carried out in these configurations:

Linux Windows macOS
Machine Dell Latitude 5520 (2022) Machine Dell Latitude 5520 (2022) Machine Apple MacBook Pro (13-inch, 2022)
CPU Intel Core i5-1145G7 @ 2.60 GHz CPU Intel Core i5-1145G7 @ 2.60 GHz CPU Apple M2
RAM 16 GiB RAM 16 GiB RAM 16 GiB
GPU Intel® Iris® Xe Graphics GPU Intel® Iris® Xe Graphics GPU Apple M2
Screen resolution 1920x1080 Screen resolution 1920x1080 Screen resolution 2560x1600
Host OS Ubuntu Desktop 22.04 LTS Host OS Windows 11 Pro 23H2 Host OS macOS Sonoma 14.5
Architecture x86-64 Architecture x86-64 Architecture arm64
Native Firefox v125.0.3 Native Firefox v125.0.3 Native Firefox v125.0.3
Docker image Ubuntu Server 22.04 LTS Docker image Ubuntu Server 22.04 LTS Docker image Ubuntu Server 22.04 LTS
Docker image architecture x86-64 Docker image architecture x86-64 Docker image architecture arm64
Containerised Firefox v125.0.3 Containerised Firefox v125.0.3 Containerised Firefox v125.0.3
Docker Engine v26.1.4 Docker Desktop v4.30.0 Docker Desktop v4.30.0
Docker Compose v2.27.1
Host Wayland compositor Mutter v42.9 Host Wayland compositor Weston (provided by WSLg) GitHub commit f227edd6 Host Wayland compositor N/A
Host X server (X.Org/XWayland) Host X server (XWayland) Host X server (XQuartz) v2.8.5
Host PulseAudio server PulseAudio (on PipeWire 0.3.48) Host PulseAudio server (provided by WSLg) GitHub commit 6f045ff0 Host PulseAudio server v17.0
WSL 2 v2.2.4.0
WSL 2 Linux Kernel with camera driver support v5.15.153.1 - Download
WSLg v1.0.61

Please read the section Performance evaluation of the containerised solution of the submission.

The Python scripts used to calculate the confidence intervals and ratios and to plot the results can be found in this folder: 2-performances.

To plot the graphs, run this command inside the 2-performances folder:

python3 ./graphs.py

The raw data are available in the following folder: preventing-webrtc-ip-address-leaks/3-raw-data/2-performance-data.

We use three open-source benchmark suites actively developed by the three main web browser engine developers:

  • Motionmark v1.3 [1, 2], graphics benchmark, open governance [3], jointly developed by the developers of Blink/V8 (Chromium & derivatives), Gecko/SpiderMonkey (Firefox) and WebKit/JavaScriptCore (Safari);
  • Speedometer v3.0 [3, 4], web application responsiveness benchmark, open governance [6], jointly developed by the developers of Blink/V8 (Chromium & derivatives), Gecko/SpiderMonkey (Firefox) and WebKit/JavaScriptCore (Safari);
  • JetStream2 v2.2 [7], JavaScript/WebAssembly benchmark suite governed by Apple alone [8], including benchmarks developed by Apple, Mozilla and Google. As this suite is managed by Apple: Mozilla and Google retrieve the latest suite releases separately from Apple's main repository: https://github.com/WebKit/WebKit/tree/main/PerformanceTests/JetStream2 [Accessed: Feb. 26, 2024].

We use the browserbench.org website managed by Apple, which publicly hosts these three benchmark suites.

References

[1] S. Fraser, D. Jackson, and N. Ryosuke, About MotionMark 1.3, MotionMark. Accessed: Feb. 26, 2024. [Online]. Available: https://browserbench.org/MotionMark/about.html.
[2] S. Fraser et al., MotionMark. Jan. 10, 2024. Accessed: Feb. 26, 2024. [Online]. Available: https://github.com/WebKit/MotionMark.
[3] M. Maxfield, MotionMark Moves to Open Governance, WebKit. Accessed: Feb. 26, 2024. [Online]. Available: https://webkit.org/blog/14359/motionmark-moves-to-open-governance/.
[4] Speedometer developers, About Speedometer 3.0, Speedometer 3.0. Accessed: June 3, 2024. [Online]. Available: https://browserbench.org/Speedometer3.0/.
[5] Speedometer developers, Speedometer. The WebKit Open Source Project, Feb. 20, 2024. Accessed: Feb. 26, 2024. [Online]. Available: https://github.com/WebKit/Speedometer.
[6] N. Ryosuke and T. Kober, Speedometer/Governance.md, Speedometer/Governance.md at main · WebKit/Speedometer. Accessed: Feb. 26, 2024. [Online]. Available: https://github.com/WebKit/Speedometer/blob/main/Governance.md.
[7] JetStream 2 developers, JetStream 2 In-Depth Analysis, JetStream 2 In-Depth Analysis. Accessed: Feb. 26, 2024. [Online]. Available: https://browserbench.org/JetStream/in-depth.html.
[8] S. Barati and M. Saboff, Introducing the JetStream 2 Benchmark Suite, WebKit. Accessed: Feb. 26, 2024. [Online]. Available: https://webkit.org/blog/8685/introducing-the-jetstream-2-benchmark-suite/.