[Bug] Concurrent archivers count wrong when starting console through a wrapper shell script

[mpdude]

What happened?

I need to start console core:archive ... through a wrapper shell script. This is to redirect stdout and stderr into the syslog through the logger command. Also, I use time to gather more information about the process, the amount of memory needed etc.

In order to find the number of running archivers, Matomo looks (here:

matomo/core/CronArchive.php

Line 1449 in a954f3e

private function hasReachedMaxConcurrentArchivers()

) at the list of all running processes of the current UID (ps wwx) and does a simple string matching against the process commands, looking for core:archive, console and --matomo-domain.

Due to a hierarchy of bash shells being started in my case, a single running archiver may be counted as five to seven instances, depending on how I write my wrapper script.

I think this could be fixed if the process list would restricted to processes that are process group leaders.

What should happen?

Every archiver instance should only be counted once.

How can this be reproduced?

Create a wrapper shell script logwrap like this:

#!/bin/bash

NAME=$1[$$]
exec 2> >(exec logger --skip-empty --priority err --tag $NAME) > >(exec logger --skip-empty --priority notice --tag $NAME)

echo "Starting $NAME"

shift
sh -c "$*"

echo "process terminated with exit_code=$?"

Then, start the archiver as

./logwrap matomo:archive-reports time --output=/dev/stdout -vvv www/console core:archive ...

The purpose of logwrap is to redirect all stderr and stdout of a program to syslog, marking the log lines with a given tag ("matomo:archive-reports") and use different log levels for output and errors.

time will start the given command as a subprocess as well and print metrics after it finished.

All this leads to a hierarchy of processes in the ps wwx output that all contain the strings console, core:archive etc. somewhere in the process name, altough only one of the processes ultimately is the running archiver.

If you want to see the resulting process hierarchy, add system("ps af"); at the top of www/console. It will print the hierarchy of processes at the time the console is started. It looks similar to this:

     PID TTY      STAT   TIME COMMAND
 1615802 pts/0    Ss     0:00 -bash
 1625443 pts/0    S+     0:00  \_ /bin/bash logwrap matomo:archive-reports time --output=/dev/stdout -vvv ... www/console core:archive ...
 1625444 pts/0    S+     0:00      \_ logger --skip-empty --priority err --tag matomo:archive-reports[1625443]
 1625445 pts/0    S+     0:00      \_ logger --skip-empty --priority notice --tag matomo:archive-reports[1625443]
 1625446 pts/0    S+     0:00      \_ sh -c /usr/bin/time --output=/dev/stdout -vvv www/console core:archive ...
 1625447 pts/0    S+     0:00          \_ /usr/bin/time --output=/dev/stdout -vvv www/console core:archive ...
 1625448 pts/0    S+     0:00              \_ /usr/bin/php www/console core:archive ...
 1625449 pts/0    S+     0:00                  \_ sh -c ps af
 1625450 pts/0    R+     0:00                      \_ ps af

The problem could be avoided if the process list would be restricted to those processes that are process group leaders.

For example, system("ps wwx -o pid,pgid,cmd | awk '$1 == $2'"); would make ps show the process PID, the process group leader PID and the command, and then use awk to keep only lines where the PID matches the process group leader PID. In the above example, that would be the process with PID 1625443.

Matomo version

5.2.1

PHP version

8.3.15

Server operating system

Ubuntu 22.04

What browsers are you seeing the problem on?

Not applicable (e.g. an API call etc.)

Computer operating system

Mac OS

Relevant log output

No response

Validations

• Read our Contributing Guidelines.
• Follow our Security Policy.
• Check that there isn't already an issue that reports the same bug to avoid creating duplicates.
• The provided steps to reproduce is a minimal reproducible of the Bug.