From 5974cbf7b6dc98c62cdb21866c96abddf186f63b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alejandro=20L=C3=B3pez?= Date: Fri, 8 Dec 2023 15:15:09 +0100 Subject: [PATCH] KCM: Document KCM debugging Added some KCM-specific details to the already present debug information. --- src/contrib/debugging-sssd.rst | 103 +++++++++++++++++++++++++++++++-- src/troubleshooting/basics.rst | 2 +- 2 files changed, 98 insertions(+), 7 deletions(-) diff --git a/src/contrib/debugging-sssd.rst b/src/contrib/debugging-sssd.rst index 907f842..f3f3b34 100644 --- a/src/contrib/debugging-sssd.rst +++ b/src/contrib/debugging-sssd.rst @@ -20,6 +20,12 @@ Child processes data. They are located in ``/usr/libexec/sssd`` and suffixed with ``_child``, for example ``krb5_child``, ``ldap_child`` or ``gpo_child``. +KCM + ``sssd-kcm`` is a part of SSSD that works independently of the other + processes. It is a store for Kerberos credentials and is usually activated + by ``systemd`` when a Kerberos tool (such as ``kinit`` or ``klist``) open + its socket for communication. + The following snippet shows a list of running SSSD process with three responders (nss, pam and pac) and three different domains (ldap.test, samba.test and ipa.test). @@ -90,7 +96,6 @@ option in ``sssd.conf``, for example to 30000 seconds. .. code-block:: ini [sssd] - config_file_version = 2 services = nss, pam domains = ldap.test, samba.test, ipa.test user = root @@ -115,15 +120,15 @@ option in ``sssd.conf``, for example to 30000 seconds. timeout=30000 ... -Attaching debugger to a running process -*************************************** +Attaching a debugger to a running process +***************************************** There is only one process for each responder that can be distinguished by name, therefore it is simple to attach a debugger to the running process. For example: .. code-block:: - $ sudo gdb program `pgrep sssd_nss` + $ sudo gdb /usr/libexec/sssd/sssd_nss `pgrep sssd_nss` There can be multiple backend ``sssd_be`` processes and we need to use the ``--domain`` parameter to distinguish between them. Therefore we want to use the @@ -133,11 +138,11 @@ the debugger to ``ldap.test`` domain: .. code-block:: - $ sudo gdb program `pgrep -f "sssd_be.+ldap.test"` + $ sudo gdb /usr/libexec/sssd/sssd_be `pgrep -f "sssd_be.+ldap.test"` .. seealso:: - We created set of `gdb extensions `__ for + We created a set of `gdb extensions `__ for SSSD that provides pretty printers to some difficult SSSD structures. Debugging forked process from its start @@ -206,3 +211,89 @@ i.e. ``su admin@ipa.test``. Then start gdb and attach it to the server: debugging the child once it is forked off the parent. See `gdb manual `__ for more information. + +Debugging KCM +************* +KCM is different from other SSSD processes in the way it is launched. It is not +forked by the monitor as all the other processes, but it is launched by a +``systemd trigger unit`` when a client application (like ``klist``) opens its +socket. After 5 minutes of inactivity it will automatically shut down unless +ticket renewal is active, in which case the timeout is disabled). + +Once it is active (running) it is possible to attach a debugger as described +above, but it is also possible to launch it manually (instead of waiting for +``systemd`` to trigger it). A ``systemd trigger`` is a special type of unit that +will start a ``systemd service`` when a certain event happens. For KCM the event +is a client opening the socket. The service is ``sssd-kcm.service`` and is +triggered by ``sssd-kcm.socket``. + +To manually launch KCM, start the service: + +.. code-block:: console + + $ sudo systemctl start sssd-kcm.service + +It is also possible to stop it with the ``stop`` command. + +While debugging, it can be useful to disable the automatic shut down by setting +the ``responder_idle_timeout`` option to 0 in ``sssd.conf``. Similarly, the +``client_idle_timeout`` option should be set to a large number of seconds. +This value specifies how long a client can sit idle before being disconnected. + +.. code-block:: ini + + [kcm] + responder_idle_timeout=0 + client_idle_timeout=10000 + +KCM can also be launched from the command line. For that, disable the +``systemd trigger`` and stop any active instance of KCM. + +.. code-block:: console + + $ sudo systemctl stop sssd-kcm.socket + $ sudo systemctl stop sssd-kcm.service + +If you need to watch KCM's start up, launch it from the command line with +some useful debug options: + +.. code-block:: console + + $ sudo /usr/libexec/sssd/sssd_kcm --uid 0 --gid 0 --logger=stderr --debug-level=9 + +To debug KCM from its start, launch ``sssd_kcm`` in the debugger: + +.. code-block:: console + + $ sudo gdb /usr/libexec/sssd/sssd_kcm + (gdb) break main + (gdb) run --uid 0 --gid 0 --logger=files + +Alternatively, launch ``sssd_kcm`` with ``strace``: + +.. code-block:: console + + $ sudo strace /usr/libexec/sssd/sssd_kcm --uid 0 --gid 0 --logger=files + +or, to dump of the KCM socket data (protocol request/response), intermixed with +KCM debug logging: + +.. code-block:: console + + $ sudo strace -p $(pidof sssd_kcm) -Ttxvs 1024 -e read=$FDS -e read,readv,write + + +Finally, to debug what the Kerberos tools are doing and how they interact with +KCM, you can set the ``KRB5_TRACE`` environment variable to the path of the +file that will receive the logged messages: + +.. code-block:: console + + $ export KRB5_TRACE=/dev/stdout + $ kinit + +.. seealso:: + + More information can be found in KCM's design page, but please be aware + that the information in this page is not maintained and can be obsolete. + diff --git a/src/troubleshooting/basics.rst b/src/troubleshooting/basics.rst index 31d35fe..7e76def 100644 --- a/src/troubleshooting/basics.rst +++ b/src/troubleshooting/basics.rst @@ -8,7 +8,7 @@ SSSD debug logs Each SSSD process is represented by a section in the ``sssd.conf`` config file. To enable debugging persistently across SSSD service restarts, put the directive ``debug_level=N``, where N typically stands for a number between 1 and 10 into the particular section. Debug levels up to 3 should log mostly failures and anything above level 8 provides a large number of log messages. Level 6 might be a good starting point for debugging problems. You can also use the ``sss_debuglevel(8)`` tool to enable debugging on the fly without having to restart the daemon. -On Fedora/RHEL, debug logs are stored under ``/var/log/sssd``. There is one log file per SSSD process. The services (also called responders) log into a log file called ``sssd_$service``, for example NSS responder logs to ``/var/log/sssd/sssd_nss.log``. Domain sections log to files called ``sssd_$domainname.log``. The short-lived helper processes also log to their own log files, such as ``ldap_child.log`` or ``krb5_child.log``. +On Fedora/RHEL, debug logs are stored under ``/var/log/sssd``. There is one log file per SSSD process. The services (also called responders) log into a log file called ``sssd_$service``, for example NSS responder logs to ``/var/log/sssd/sssd_nss.log``. Domain sections log to files called ``sssd_$domainname.log``. The short-lived helper processes also log to their own log files, such as ``ldap_child.log`` or ``krb5_child.log``. KCM logs to the file ``sssd_kcm.log``. .. note::