Friday, September 9, 2022

Sysmon For Linux - where it works and where not?

I was installing Sysmon For Linux with use of the project instructions.
Once sysmon is installed, it requires to accept EULA and start with some configuration - see relevant paragraph in the project documentation.

Let me describe general rules before going into details:

  1. AWS was used to create following linux machines.
    AMIs owned by AWS or organisation that is responsible for specific distribution were selected only.
    All labs were created in us-west-1
  2. The same config files was used for all distributions/versions
  3. The effort was a part of greater automation project. As a consequence, some commands were modified to non-interactive/pre-approved.
  4. If there was an issue, no more than few hours hours were spent to find a solution.
    If the solution would require to use different than Microsoft package for specific distribution/version, the solution is not applicable.

Let me start from distributions where the instructions were sufficient:

  • Ubuntu 18.04 and 20.04
  • openSUSE 15
  • Fedora 33 - just one comment here: wget was not pre-installed, so before the instructions, following has to be executed:
sudo dnf search wget
sudo dnf install -y wget
There was one case (for SLES 15), where the instruction had to be modified to make the sysmon works. A solution was to take openSUSE instruction and instead of pointing to openSUSE 15 repo, to point to relevant SLES repo.
This can be also a hint for solution for other cases - see Ubuntu 22 as an example.
... and the code:
sudo zypper install -y libicu
sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
wget -q https://packages.microsoft.com/config/sles/15/prod.repo
sudo mv prod.repo /etc/zypp/repos.d/microsoft-prod.repo
sudo chown root:root /etc/zypp/repos.d/microsoft-prod.repo

sudo zypper install -y sysmonforlinux

Let's come to cases where the instruction failed:

  • Ubuntu 21.04 - it's already EoL; administrator is forced to upgrade to Ubuntu 22.04
  • Ubuntu 22.04
sudo apt-get install sysmonforlinux

ends with error

Unable to locate package sysmonforlinux

If package for Ubuntu 21.04 is used all seems to work fine - sysmon was started and all of 8 Event IDs are returned. Content also looks fine at first glance, but I'm not sure about details.

  • Fedora 34
[root@ip-172-31-10-70 fedora]# sysmon -accepteula -i /tmp/all_rules_included.xml 

Sysmon v1.0.2 - Monitors system events
Sysinternals - www.sysinternals.com
By Mark Russinovich, Thomas Garnier and Kevin Sheldrake
Copyright (C) 2014-2021 Microsoft Corporation
Using libxml2. libxml2 is Copyright (C) 1998-2012 Daniel Veillard. All Rights Reserved.

Loading configuration file with schema version 4.60
Sysmon schema version: 4.81
Configuration file validated.
Job for sysmon.service failed because the control process exited with error code.
See "systemctl status sysmon.service" and "journalctl -xeu sysmon.service" for details.
[root@ip-172-31-10-70 fedora]# systemctl status sysmon.service
× sysmon.service - Sysmon event logger
     Loaded: loaded (/etc/systemd/system/sysmon.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Fri 2022-09-09 10:36:37 UTC; 5s ago
    Process: 15510 ExecStart=/opt/sysmon/sysmon -i /opt/sysmon/config.xml -service (code=exited, status=13)
        CPU: 5.781s

Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal sysmon[15557]: /opt/sysinternalsEBPF/sysinternalsEBPF_offsets.conf
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal sysmon[15510]: Telemetry failed to start: Configuration could not be loaded
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal systemd[1]: sysmon.service: Control process exited, code=exited, status=13/n/a
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal sysmon[15557]: <Event><System><Provider Name="Linux-Sysmon" Guid="{ff032593-a8d3-4f13-b0d6-01fc615a0f97}"/><EventID>4</EventID><Version>3</Version><Level>4</Level><Task>4</Task><Opcode>0</Opco>
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal sysmon[15557]: Stopping....
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal sysmon[15557]: Total events: 0, bad events: 0, ratio = -nan
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal sysmon[15557]: Lost events: 0, in 0 notifications
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal systemd[1]: sysmon.service: Failed with result 'exit-code'.
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal systemd[1]: Failed to start Sysmon event logger.
Sep 09 10:36:37 ip-172-31-10-70.us-west-1.compute.internal systemd[1]: sysmon.service: Consumed 5.781s CPU time.

The error suggests issue with the package, so no solution was tried. 

  • Debian 10 & Debian 11
    Error was the same for both versions (beside version number OFC)
fatal: [X.XXX.XX.XXX]: FAILED! => {"changed": true, "cmd": "sudo apt-get update
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install sysmonforlinux
", "delta": "0:00:03.241709", "end": "2022-09-06 10:24:14.629355", "msg": "non-zero return code", "rc": 100, "start": "2022-09-06 10:24:11.387646", "stderr": "W: GPG error: https://packages.microsoft.com/debian/10/prod buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
E: The repository 'https://packages.microsoft.com/debian/10/prod buster InRelease' is not signed.
W: GPG error: https://packages.microsoft.com/debian/10/prod buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
E: The repository 'https://packages.microsoft.com/debian/10/prod buster InRelease' is not signed.
E: Unable to locate package sysmonforlinux", "stderr_lines": ["W: GPG error: https://packages.microsoft.com/debian/10/prod buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF", "E: The repository 'https://packages.microsoft.com/debian/10/prod buster InRelease' is not signed.", "W: GPG error: https://packages.microsoft.com/debian/10/prod buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF", "E: The repository 'https://packages.microsoft.com/debian/10/prod buster InRelease' is not signed.", "E: Unable to locate package sysmonforlinux"], "stdout": "Hit:1 http://security.debian.org/debian-security buster/updates InRelease
Hit:2 http://cdn-aws.deb.debian.org/debian buster InRelease
Hit:3 http://cdn-aws.deb.debian.org/debian buster-updates InRelease
Get:4 https://packages.microsoft.com/debian/10/prod buster InRelease [29.8 kB]
Err:4 https://packages.microsoft.com/debian/10/prod buster InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
Hit:5 http://cdn-aws.deb.debian.org/debian buster-backports InRelease
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
apt-transport-https is already the newest version (1.8.2.3).
0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
Hit:1 http://security.debian.org/debian-security buster/updates InRelease
Hit:2 http://cdn-aws.deb.debian.org/debian buster InRelease
Hit:3 http://cdn-aws.deb.debian.org/debian buster-updates InRelease
Hit:4 http://cdn-aws.deb.debian.org/debian buster-backports InRelease
Get:5 https://packages.microsoft.com/debian/10/prod buster InRelease [29.8 kB]
Err:5 https://packages.microsoft.com/debian/10/prod buster InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...", "stdout_lines": ["Hit:1 http://security.debian.org/debian-security buster/updates InRelease", "Hit:2 http://cdn-aws.deb.debian.org/debian buster InRelease", "Hit:3 http://cdn-aws.deb.debian.org/debian buster-updates InRelease", "Get:4 https://packages.microsoft.com/debian/10/prod buster InRelease [29.8 kB]", "Err:4 https://packages.microsoft.com/debian/10/prod buster InRelease", "  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF", "Hit:5 http://cdn-aws.deb.debian.org/debian buster-backports InRelease", "Reading package lists...", "Reading package lists...", "Building dependency tree...", "Reading state information...", "apt-transport-https is already the newest version (1.8.2.3).", "0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.", "Hit:1 http://security.debian.org/debian-security buster/updates InRelease", "Hit:2 http://cdn-aws.deb.debian.org/debian buster InRelease", "Hit:3 http://cdn-aws.deb.debian.org/debian buster-updates InRelease", "Hit:4 http://cdn-aws.deb.debian.org/debian buster-backports InRelease", "Get:5 https://packages.microsoft.com/debian/10/prod buster InRelease [29.8 kB]", "Err:5 https://packages.microsoft.com/debian/10/prod buster InRelease", "  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF", "Reading package lists...", "Reading package lists...", "Building dependency tree...", "Reading state information..."]}

There were two solutions applied, but none resolved the issue

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com EB3E94ADBE1229CF
sudo apt update 

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -

No comments: