Add support for ESXI auth, hostd and shell logs. #1385

william-billaud · 2025-11-03T16:48:06Z

Add support for three ESXi logs with related test data from ESXi 6/7/8/9:

auth.log
shell.log
hostd.log

I may take a look on some others files later (vodb, vmk*, vpxa...) but it may be easier to validate the structure before adding more files. I'm not sure the plugin inheritance is the best solution, but if each log type is just an entry of the same plugins, is that we can't have a check_compatible function by log type.

One of the main problems is that even if most files seems to have the same format, some small differences exists (e.g in shell.log). Regarding plugins name I'm not really sure how to work with name conflict (eg : auth).

Regarding log file location, as it may change regarding version and how log were collected (ESXi does some weird thing with symlinks) plugins look for usual location, as well as in the osdata partition.

Log location may also be found in the /etc/vmsyslog.conf but I have not implemented, neither worked on how it works.

Furthermore :

Add files related to pycharm, uv and coverage to .gitignore

This issue partially resolve #1218

.gitignore

codspeed-hq · 2025-12-04T15:56:22Z

CodSpeed Performance Report

Merging #1385 will not alter performance

_{Comparing william-billaud:esxi_hostd_log (78e369a) with main (52e9f84)}

Summary

✅ 9 untouched

dissect/target/plugins/os/unix/esxi/_os.py

Schamper · 2026-01-08T15:52:24Z

dissect/target/plugins/os/unix/esxi/log.py

+)
+
+
+class EsxiLogBasePlugin(Plugin, ABC):


Would you mind changing this around to a more functional approach, rather than ABC? So e.g. a function in the global scope find_logs(target: Target, name: str) or something like that. As well the parser code.

done. Let me know if it's OK for you

dissect/target/plugins/os/unix/esxi/log.py

Schamper · 2026-01-08T15:55:21Z

dissect/target/plugins/os/unix/esxi/log.py

+    """Esxi base log plugin."""
+
+    __register__ = False
+    COMMON_LOG_LOCATION: ClassVar[list[str]] = ["/var/log", "/var/run/log", "/scratch/log", "/var/lib/vmware/osdata"]


Isn't this mostly redundant? As far as I know, basically all of these symlink to the same location (the OSDATA volume). Taking just /var/log should be enough?

Also the reason why I mention that the osdata_fs internal is unnecessary, as it should already be properly symlinked by the OS plugin and available/resolvable through the symlinks.

I will take a deeper look, but it's maybe challenging to have something working for old and new storage format : https://www.vmware.com/docs/esxi-system-storage-changes

Furthermore I also want the plugins to work with uac and tech support, in addition to disk (I'm not aware of another use case).

Basically after storage change the structure on a live system is the following for ESXi 7+ (tested on ESXi 8 and 9, I must find an ESXi 7 for test

/scratch -> symlink to the osdata volume (/vmfs/volume/OSDATA-XXX)
/var/run/log -> symlink to /scratch/log (so with transitivity to /vmfs/volume/OSDATA-XXX/log
/var/log -> some file (file by file, not the whole folder) are symlink to /var/run/log, but this folder does not contains compressed log file (e.g symlink for auth.log but not auth.0.gz)

But when working with tech support, log are located only in /var/run/log

And when working with UAC collection, log are in /scratch/log (with symlink from /var/log/auth.log)

I have checked on UAC from ESXi 6 and 7, log were also on /scratch/log

Maybe I should modify the loader to create osdata symlink to /scratch (and from /var/run/log to /scratch/log) when working with raw disk or equivalent but we would still have an issue with either UAC or tech support (or we should at least check both location?)

I will make more test next week

I have made some modification to create a symlink from osdata to /scratch if it doesn't exist, I will test it with older version next week and let you know when it's OK.

william-billaud · 2026-01-12T15:24:19Z

I have modified symlink made by ESXi, especially related to OS data partition detection. Thus this plugin no longer need to identify OSdata volume.
I'm not sure if current implementation was really working on this point (as vmfs is a lvm, thus OSDATA is a Physical volume). I have kept the previous logic working.

@Schamper let me know if it's OK for you; I can send the 3 disk if you want, with associated uac and techsupport

TLDR :

On my freshly installed esxi 8 and esxi 9, Volume with name OSDATA is empty, as this is a physical volume, volume with content is a logical volume with label osdata-{uuid}.
issue with vfat on ESXi 6 (but this is something for another issue)

Details

With a slightly modified version of target info that display volume offset + vmfs label + if volume is an lvm if available

Freshly installed ESXI 8

uv run --extra full target-info  ~/Documents/VM/esxi_8
2026-01-12T14:45:58.497393Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_8>: local.tgz is encrypted but static decryption failed and no dynamic decryption available! [dissect.target.target]
2026-01-12T14:45:58.504685Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_8>: No locker.conf! [dissect.target.target]
2026-01-12T14:45:58.539669Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_8>: No configstore found, unable to mount NFS shares [dissect.target.target]
<Target /home/vprb6956/Documents/VM/esxi_8>

Disks
- <Disk type='qcow2' size=32212254720>

Volumes
- <Volume name='BOOT' offset=32768 size=104857088 fs='fat' label=None is_lvm=False>
- <Volume name='BOOTBANK1' offset=106954752 size=4293918208 fs='fat' label=None is_lvm=False>
- <Volume name='BOOTBANK2' offset=4401922048 size=4293918208 fs='fat' label=None is_lvm=False>
- <Volume name='OSDATA' offset=8696889344 size=23515347968 fs=None label=None is_lvm=True>
- <Volume name='69007f54-39ba5b15-9cd4-5254002bf7f4' offset=None size=23353884672 fs='vmfs' label='OSDATA-69007f54-4071b795-d663-5254002bf7f4' is_lvm=False>

Mounts
- <Mount fs='fat' path='/vmfs/volumes/defa3a97-6642d5f1-74b6-952bf3cd2b76'>
- <Mount fs='fat' path='/vmfs/volumes/ffec8640-3e4bafb3-1099-0033e073e989'>
- <Mount fs='fat' path='/vmfs/volumes/b48d2913-2b4a5b02-4baf-4bb1e50a53c3'>
- <Mount fs='vmfs' path='/vmfs/volumes/69007f54-4071b795-d663-5254002bf7f4'>

Hostname       : localhost
Domain         : None
Ips            :
Os family      : esxi
Os version     : VMware ESXi 8.0.3-0.70.24677879
Architecture   : x86_64-esxi
Language       :
Timezone       : None
Install date   : None
Last activity  : 2025-04-03T07:45:28.000000+00:00

Freshly installed ESXI 9

 uv run --extra full target-info  ~/Documents/VM/esxi_9
2026-01-12T14:49:53.043050Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_9>: local.tgz is encrypted but static decryption failed and no dynamic decryption available! [dissect.target.target]
2026-01-12T14:49:53.052286Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_9>: No locker.conf! [dissect.target.target]
2026-01-12T14:49:53.081220Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_9>: No configstore found, unable to mount NFS shares [dissect.target.target]
<Target /home/vprb6956/Documents/VM/esxi_9>

Disks
- <Disk type='qcow2' size=32212254720>

Volumes
- <Volume name='BOOT' offset=32768 size=104857088 fs='fat' label=None is_lvm=False>
- <Volume name='BOOTBANK1' offset=106954752 size=4293918208 fs='fat' label=None is_lvm=False>
- <Volume name='BOOTBANK2' offset=4401922048 size=4293918208 fs='fat' label=None is_lvm=False>
- <Volume name='OSDATA' offset=8696889344 size=23515347968 fs=None label=None is_lvm=True>
- <Volume name='69612a7d-271bdc46-b890-525400dce313' offset=None size=23353884672 fs='vmfs' label='OSDATA-69612a7d-2c879838-ff95-525400dce313' is_lvm=False>

Mounts
- <Mount fs='fat' path='/vmfs/volumes/fafc4ace-8b4ac265-87bd-57c0ed340c4a'>
- <Mount fs='fat' path='/vmfs/volumes/5cd6b76c-d74cd64d-b685-57462e717706'>
- <Mount fs='fat' path='/vmfs/volumes/087ab5b5-df4084c7-e69c-b28ee4e289dd'>
- <Mount fs='vmfs' path='/vmfs/volumes/69612a7d-2c879838-ff95-525400dce313'>

Hostname       : localhost
Domain         : None
Ips            :
Os family      : esxi
Os version     : VMware ESXi 9.0.0-0.24678710
Architecture   : x86_64-esxi
Language       :
Timezone       : None
Install date   : None
Last activity  : 2000-01-01T00:00:00.000000+00:00

Freshly installed ESXI 6

uv run --extra full target-info  ~/Documents/VM/esxi_6
2026-01-12T14:50:24.630783Z [warning  ] Failed to detect <class 'dissect.target.volumes.bde.BitlockerVolumeSystem'> volume system [dissect.target.volume]
2026-01-12T14:50:24.631111Z [warning  ] Failed to detect <class 'dissect.target.volumes.bde.BitlockerVolumeSystem'> volume system [dissect.target.volume]
2026-01-12T14:50:24.631463Z [warning  ] Failed to detect <class 'dissect.target.volumes.bde.BitlockerVolumeSystem'> volume system [dissect.target.volume]
2026-01-12T14:50:24.631646Z [warning  ] Failed to detect <class 'dissect.target.volumes.bde.BitlockerVolumeSystem'> volume system [dissect.target.volume]
2026-01-12T14:50:24.883151Z [warning  ] Failed to detect fat filesystem [dissect.target.filesystem]
2026-01-12T14:50:24.883505Z [warning  ] Failed to detect fat filesystem [dissect.target.filesystem]
2026-01-12T14:50:24.883743Z [warning  ] Failed to detect fat filesystem [dissect.target.filesystem]
2026-01-12T14:50:24.883978Z [warning  ] Failed to detect fat filesystem [dissect.target.filesystem]
2026-01-12T14:50:25.889336Z [warning  ] <Target /home/<CENSORED>/Documents/VM/esxi_6>: No configstore found, unable to mount NFS shares [dissect.target.target]
<Target /home/<CENSORED>/Documents/VM/esxi_6>

Disks
- <Disk type='qcow2' size=32212254720>

Volumes
- <Volume name='part_00008000' offset=32768 size=4161024 fs='fat' label=None is_lvm=False>
- <Volume name='part_d8400000' offset=3628072960 size=4293918208 fs='fat' label=None is_lvm=False>
- <Volume name='part_1d8300000' offset=7921991680 size=24290245632 fs=None label=None is_lvm=True>
- <Volume name='part_00404000' offset=4210688 size=262127104 fs='fat' label=None is_lvm=False>
- <Volume name='part_0fe04000' offset=266354688 size=262127104 fs='fat' label=None is_lvm=False>
- <Volume name='part_1f804000' offset=528498688 size=115326464 fs=None label=None is_lvm=False>
- <Volume name='part_26604000' offset=643842048 size=299875840 fs='fat' label=None is_lvm=False>
- <Volume name='part_38400000' offset=943718400 size=2684354048 fs=None label=None is_lvm=False>
- <Volume name='part_1804158fa00' offset=1650363791872 size=126300121088 fs=None label=None is_lvm=False>
- <Volume name='part_339a2d9a00' offset=221630011904 size=397150115328 fs=None label=None is_lvm=False>
- <Volume name='part_dede858600' offset=957216032256 size=982645794816 fs=None label=None is_lvm=False>
- <Volume name='part_8b0000000' offset=37312528384 size=33554432 fs=None label=None is_lvm=False>
- <Volume name='6964fc45-9ba0ec50-cf85-525400ed184f' offset=None size=24159191040 fs='vmfs' label='datastore1' is_lvm=False>


Mounts
- <Mount fs='fat' path='/vmfs/volumes/7e16c2b3-41117583-a494-7b468ad04e89'>
- <Mount fs='fat' path='/vmfs/volumes/6964fc45-b0efa800-cc75-525400ed184f'>
- <Mount fs='fat' path='/vmfs/volumes/e29b8575-2024999d-4ad9-f131b21b883f'>
- <Mount fs='fat' path='/vmfs/volumes/d4566d7f-7cde8e57-da7a-49ba510387d7'>
- <Mount fs='fat' path='/vmfs/volumes/6964fc3f-12a9c5b8-c1e9-525400ed184f'>
- <Mount fs='vmfs' path='/vmfs/volumes/6964fc45-a0f0af8e-8f17-525400ed184f'>

Hostname       : localhost
Domain         : None
Ips            : 192.168.122.133
Os family      : esxi
Os version     : VMware ESXi 6.7.0-14320388
Architecture   : x86_64-esxi
Language       :
Timezone       : None
Install date   : None
Last activity  : 2026-01-12T14:50:16.000000+00:00

On the same ESXi 6, result of an ls on '/' from the live system.

[root@localhost:~] ls -alh
total 805
drwxr-xr-x    1 root     root         512 Jan 12 14:13 .
drwxr-xr-x    1 root     root         512 Jan 12 14:13 ..
-rw-------    1 root     root         114 Jan 12 14:39 .ash_history
-r--r--r--    1 root     root          20 Aug  5  2019 .mtoolsrc
lrwxrwxrwx    1 root     root          49 Jan 12 14:13 altbootbank -> /vmfs/volumes/d4566d7f-7cde8e57-da7a-49ba510387d7
drwxr-xr-x    1 root     root         512 Jan 12 14:13 bin
lrwxrwxrwx    1 root     root          49 Jan 12 14:13 bootbank -> /vmfs/volumes/e29b8575-2024999d-4ad9-f131b21b883f
-r--r--r--    1 root     root      334.3K Aug  5  2019 bootpart.gz
-r--r--r--    1 root     root      221.3K Aug  5  2019 bootpart4kn.gz
drwxr-xr-x   16 root     root         512 Jan 12 14:51 dev
drwxr-xr-x    1 root     root         512 Jan 12 14:13 etc
drwxr-xr-x    1 root     root         512 Jan 12 14:13 lib
drwxr-xr-x    1 root     root         512 Jan 12 14:13 lib64
-r-x------    1 root     root       13.3K Jan 12 14:03 local.tgz
lrwxrwxrwx    1 root     root           6 Jan 12 14:13 locker -> /store
drwxr-xr-x    1 root     root         512 Jan 12 14:13 mbr
drwxr-xr-x    1 root     root         512 Jan 12 14:13 opt
drwxr-xr-x    1 root     root      128.0K Jan 12 14:51 proc
lrwxrwxrwx    1 root     root          29 Jan 12 14:13 productLocker -> /locker/packages/vmtoolsRepo/
lrwxrwxrwx    1 root     root           4 Aug  5  2019 sbin -> /bin
lrwxrwxrwx    1 root     root          49 Jan 12 14:13 scratch -> /vmfs/volumes/6964fc45-b0efa800-cc75-525400ed184f
lrwxrwxrwx    1 root     root          49 Jan 12 14:13 store -> /vmfs/volumes/6964fc3f-12a9c5b8-c1e9-525400ed184f
drwxr-xr-x    1 root     root         512 Jan 12 14:13 tardisks
drwxr-xr-x    1 root     root         512 Jan 12 14:13 tardisks.noauto
drwxrwxrwt    1 root     root         512 Jan 12 14:15 tmp
drwxr-xr-x    1 root     root         512 Jan 12 14:13 usr
drwxr-xr-x    1 root     root         512 Jan 12 14:13 var
drwxr-xr-x    1 root     root         512 Jan 12 14:13 vmfs
drwxr-xr-x    1 root     root         512 Jan 12 14:13 vmimages
lrwxrwxrwx    1 root     root          18 Aug  5  2019 vmupgrade -> /locker/vmupgrade/

william-billaud · 2026-01-13T13:42:02Z

The more I dig into the code, the more issues I find.
I'm not sure if they should be processed here.

the configstore plugins wont works with live collection (uac/techsupport) as it only check for location as found on a disk, but this is not the one seen on a live system (

dissect.target/dissect/target/plugins/os/unix/esxi/configstore.py

Line 29 in 4b7bddc

if (path := target.fs.path("/var/lib/vmware/configstore/backup/current-store-1")).exists():

).
The link log dir function (

dissect.target/dissect/target/plugins/os/unix/esxi/_os.py

Line 386 in 4b7bddc

def _link_log_dir(target: Target, cfg: dict[str, str], plugin_obj: ESXiPlugin) -> None:

) in charge of creating symlink (e.g /var/log -> /scratch/log) use the ._configstore object, which no longer exists. But as this function is called when plugin is not totally initialized, I was not able to use the configstore plugin (who probably replace de _configstore object).

I think that these issue should be processed in dedicated issue, but maybe you prefer to fix these issue here.
Even if checking for 3 differents log location is not the perfect solution, properly symlink log folder for multiple ESXi version (disk or live collection) will probably take a lot of time.

william-billaud · 2026-01-16T15:56:12Z

I Will try to fix the ESXi PLugins in a dedicated PR next week, especially regarding symlink of logs folder, for disk and live collection : uac/vmsupport/ acquire? (not sure I will have time to generate a live collection with this one).

…1509

william-billaud and others added 24 commits October 30, 2025 17:46

Add esxi hostd plugins (6&7)

3e4e6f9

Support esxi 8/9

85f03ee

Fix tests

3e084b3

Refactoring plugin

eb16998

move log files

79c46d8

Format

42fbd66

Fix record type inconsistency

5f0bc2c

Auth

ea2dde8

Merge branch 'main' into esxi_hostd_log

95b1fa3

Read hostd files

d8676d0

Change path

53b061d

Add log auth

bbceab4

Add support for auth.log

d308218

Add shell.log ESXi parsing

0e3f0ac

Remove unused code

27ef6d6

typo

9a8d5fd

Fix gitignore

60b8c4b

Typo

840254a

Documentation

d5e97de

Merge branch 'main' into esxi_hostd_log

aaebb49

Merge branch 'main' into esxi_hostd_log

3d9d773

Fix podt merge

62f9d83

Merge branch 'main' into esxi_hostd_log

c78a718

Merge branch 'main' into esxi_hostd_log

78e369a

Schamper requested changes Dec 4, 2025

View reviewed changes

.gitignore Show resolved Hide resolved

william-billaud and others added 3 commits December 4, 2025 17:25

Revert some change on .gitignore

38514b9

Merge branch 'main' into esxi_hostd_log

d271b8f

Merge branch 'main' into esxi_hostd_log

9db79c0

william-billaud requested a review from Schamper December 29, 2025 09:36

william-billaud added 2 commits December 29, 2025 10:36

Merge branch 'main' into esxi_hostd_log

bceb2b5

Merge branch 'main' into esxi_hostd_log

1ab065c

Schamper requested changes Jan 8, 2026

View reviewed changes

william-billaud and others added 4 commits January 9, 2026 15:10

Merge branch 'main' into esxi_hostd_log

837320c

Refactor ESXi log plugins into something more functional

d78dad6

Merge branch 'main' into esxi_hostd_log

1e011d8

Symlink osdata into /scratch in esxi os plugins.

52278f6

william-billaud and others added 2 commits January 12, 2026 17:05

Merge branch 'main' into esxi_hostd_log

a10503c

Modify comment

5ff012a

william-billaud requested a review from Schamper January 12, 2026 16:13

william-billaud mentioned this pull request Jan 13, 2026

Add support for ESXI vm support #1491

Closed

JSCU-CNI mentioned this pull request Jan 15, 2026

Fix ESXi configstore call in OS create method #1499

Merged

Merge branch 'main' into esxi_hostd_log

09f47ec

william-billaud added 2 commits January 19, 2026 17:15

Revert change related to ESXi plugin.

5237050

Revert change related to ESXi plugin.

bcd8648

william-billaud mentioned this pull request Jan 20, 2026

Improve ESXiPlugin support for recent version #1509

Open

6 tasks

Change log path to /var/run/log to match modification made in fox-it#…

03fbaeb

…1509

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for ESXI auth, hostd and shell logs. #1385

Add support for ESXI auth, hostd and shell logs. #1385

Uh oh!

william-billaud commented Nov 3, 2025

Uh oh!

Uh oh!

codspeed-hq bot commented Dec 4, 2025

Uh oh!

Uh oh!

Schamper Jan 8, 2026

Uh oh!

william-billaud Jan 9, 2026

Uh oh!

Uh oh!

Schamper Jan 8, 2026

Uh oh!

william-billaud Jan 9, 2026

Uh oh!

william-billaud Jan 9, 2026

Uh oh!

william-billaud commented Jan 12, 2026 •

edited

Loading

Uh oh!

william-billaud commented Jan 13, 2026

Uh oh!

william-billaud commented Jan 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add support for ESXI auth, hostd and shell logs. #1385

Are you sure you want to change the base?

Add support for ESXI auth, hostd and shell logs. #1385

Uh oh!

Conversation

william-billaud commented Nov 3, 2025

Uh oh!

Uh oh!

codspeed-hq bot commented Dec 4, 2025

CodSpeed Performance Report

Merging #1385 will not alter performance

Summary

Uh oh!

Uh oh!

Schamper Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

william-billaud Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Schamper Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

william-billaud Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

william-billaud Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

william-billaud commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Uh oh!

william-billaud commented Jan 13, 2026

Uh oh!

william-billaud commented Jan 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

william-billaud commented Jan 12, 2026 •

edited

Loading