Pre-processing of some options

[pawamoy]

I'm rewriting an argparsed-based CLI with Cappa. In the previous main function, I was first parsing arguments into an argparse namespace, then using some options of this namespace to configure logging.

def main(args: list[str] | None = None) -> int:
    parser = get_parser()
    opts = parser.parse_args(args=args)
    configure_logging(opts.log_level, opts.log_path, allow="pypiserver")
    if not opts.subcommand:
        parser.print_usage(file=sys.stderr)
        return 1
    return commands[opts.subcommand](opts)

After a bit of twiddling, here's how I achieved the same thing with Cappa:

# The following log options are added to every command.
@dataclass(kw_only=True)
class LogLevelPathOptions:
    """Log level and path options."""

    log_level: An[
        Literal["TRACE", "DEBUG", "INFO", "SUCCESS", "WARNING", "ERROR", "CRITICAL"],
        cappa.Arg(short="-L", long=True, parse=str.upper),
        Doc("Log level to use when logging messages."),
    ] = "INFO"

    log_path: An[
        str | None,
        cappa.Arg(short="-P", long=True),
        Doc("Write log messages to this file path. (Default: stderr)"),
    ] = None

    # This method actually configures logging with the provided CLI arguments.
    def process(self) -> None:  # noqa: D102
        configure_logging(self.log_level, self.log_path, allow="pypiserver")


# The logic for calling these `process()` methods is implemented in a mixin.
class ProcessOptions:
    def process_options(self) -> None:  # noqa: D102
        for parent in self.__class__.__mro__:
            if parent is ProcessOptions:
                continue
            if hasattr(parent, "process"):
                parent.process(self)


# Now just for convenience I declare a "common options" class
# that holds all my common options, to avoid repetition later.
# Note how it inherits my "options pre-processing" mixin
# (as well as other classes defining some options).
@dataclass(kw_only=True)
class CommonOptions(HelpOption, ConfPathOption, LogLevelPathOptions, ProcessOptions):
    """Common options for all commands."""


# Finally all my commands can inherit from `CommonOptions`,
# and call the mixin's `process_options()` method in their `__call__` method.
@cappa.command(
    name="start",
    help="Start the server.",
    description="Start the server in the background.",
)
@dataclass(kw_only=True)
class CommandServerStart(CommonOptions, DistDirOption, PortOption):
    """Command to start the server."""

    def __call__(self) -> int:  # noqa: D102
        self.process_options()
        start_server(dist_dir=self.dist_dir, port=self.port)
        return 0

And my main function doesn't have to do any special processing itself:

def main(args: list[str] | None = None) -> int:
    output = cappa.Output(error_format=f"[bold]{NAME}[/]: [bold red]error[/]: {{message}}")
    return cappa.invoke(CommandMain, argv=args, output=output, backend=cappa.backend, completion=False, help=False)

[DanCardin]

I tend to do this sort of stuff in invoke dependencies

from __future__ import annotations

from dataclasses import dataclass
from typing import Annotated as An
from typing import Literal

import cappa


def process(command: Command):
    print(f"configure_logging: {command.log_level}")


@dataclass(kw_only=True)
class Command:
    log_level: An[Literal["TRACE", "DEBUG"], cappa.Arg(short="-L")] = "INFO"
    subcommands: cappa.Subcommands[Foo | Bar]


@dataclass
class Foo:
    def __call__(self):
        print("foo")


@dataclass
class Bar:
    def __call__(self):
        print("bar")


cappa.invoke(Command, deps=[process])

If there's a direct dependence on the output in downstream deps, you can just depend on them directly, or if (with logging) you'd want to do it on all subcommands regardless of context, and dont want to explicitly call out to it in each command's impl.

---

Not relevant to this example, but something that occurs to me.

Although a potential drawback here is if the dep is only utilized in a subset of commands (e.g. git remote [set-url, list, whatever]. one level removed from the top, you'd (currently) have to resort to calling out to the function yourself. it's probably an indicator that @command(deps=[]) might be worth looking into

[pawamoy]

If you take a look at uv, it does support the same global options at any level:

uv -q python list
uv python -q list
uv python list -q

That's what I'm trying to do too 😄

[DanCardin]

Right i guess i'm just saying that my reading of your original example wouldn't behave that way because it doesn't distinguish between options at different levels, unless configure_logging is doing something internally (but i can't see how it would, because the default is INFO, so it can't tell between the default and a lack of an input?

And rereading your original comparison with argparse, I would assume it's that argparse writes all the results to a flat namespace, so whichever one is parsed last will happen to win (i assume).

And so, if I were to implement usage of Self (which i dont see a downside of, either way), then i'd suggest that, but then it would only yield/read the selected subcommand's log_level, and e.g. uv python list -q would work, but uv -q python list wouldn't do anything.

Whereas my original suggestion with command: CommandMain would be the opposite, although technically you could then traverse the command heirarchy command.subcommand.subcommand.log_level (given a normal-enough command structure) to implement a heirarchy lookup yourself.

The third (or fourth, after your original impl) option might be something like what uv/clap are ultimately using: https://docs.rs/clap/latest/clap/struct.Arg.html#method.global.

[DanCardin]

...also seems like the type checkers dont like this use of Self because it wont end up being in the context of a class definition. So, that's mildly unfortunate.

[pawamoy]

Getting inspiration from Clap sounds like a good idea ☺️ Also, yeah, I thought using Self would indeed not please type checkers. Honestly it would kinda feel like a typing hack to use Self for this use-case 🙂

In the end I'm using the initial solution I showed (process method on options, process_options method on commands, inherited thanks to a mixin class). I worried it would be annoying for reusability in other CLIs (I'm planning to use one project's command into another project, as a subcommand 🚀), but the other CLIs can just subclass the option to empty the process method, or subclass the command to empty the process_options method, so that's fine.

If you ever find an elegant way to support this use-case in a more straight-forward fashion, let me know! Otherwise it's all good for me 🙂

[DanCardin]

So fwiw https://github.com/DanCardin/cappa/tree/dc/invoke-self. I'll probably look into global options next, also. seems like that'd also solve your problem by defining it once at the top level with Arg(global=True) then it would always get written back to the root object regardless of where it's supplied.

[pawamoy]

I simplified things a bit:

@dataclass(kw_only=True)
class CommonOptions(LogLevelPathOptions, ConfPathOption, HelpOption):
    """Common options for all commands."""


@dataclass(kw_only=True)
class BaseCommand(CommonOptions):
    """Base command class."""

    def run(self) -> int:
        """Run the command."""
        raise NotImplementedError

    def __call__(self) -> int:  # noqa: D102
        configure_logging(self.log_level, self.log_path, allow="pypiserver")
        return self.run()

Now each command inherits from BaseCommand, and so will configure logging when called.

Why the run method though? you might ask. Well, it's simply so I don't have to keep calling super().__call__() in each command's __call__ method (that I now don't have to implement, instead I implement run).