CLIs with Argparse

The easiest way to create a CLI for your Settings is by decorating a function with cli():

example.py
import typed_settings as ts

@ts.settings
class Settings:
    spam: int = ts.option(default=42, help="Spam count")


@ts.cli(Settings, "example")
def cli(settings: Settings) -> None:
    """Example app"""
    print(settings)


if __name__ == "__main__":
    cli()

The cli() decorator does a few things:

  • It creates an argparse.ArgumentParser for you.

  • It uses the docstring of the decorated function as description for it.

  • It uses the default loaders (see default_loaders()) to load settings for the app "example".

  • It creates an Argparse argument for each option of the provided settings and takes default values from the loaded settings.

  • When the user invokes the CLI, it creates an updated settings instances from the argparse.Namespace.

  • It passes the settings instances to your function.

Let’s see how the generated CLI works like:

$ export EXAMPLE_SPAM=23
$ python example.py --help
usage: example.py [-h] [--spam INT]

Example app

options:
  -h, --help  show this help message and exit

Settings:
  Settings options

  --spam INT  Spam count [default: 23]
$ python example.py --spam=3
Settings(spam=3)

Note

CLI generation works with all supported settings class backends (e.g., attrs and Pydantic}).

Most examples will use the Typed Settings wrapper for attrs because the code for creating a CLI is the same. If there are notable implementation differences between the backends, the examples use inline tabs to show the code for each backend.

Tuning and Extending CLI generation

There are various ways how you can control, fine-tune and extend the default behavior of cli():

Customizing the Generated Arguments

Typed Settings tries to create the Argparse arguments in the most sensible way. But you can override all keyword arguments for add_argument() for each option individually via the argparse argument.

Lets, for example, change the generated metavar:

example.py
import typed_settings as ts


@ts.settings
class Settings:
    spam: int = ts.option(
        default=42,
        # "help" will be copied to "argparse:help"
        help="Spam count",
        argparse={"metavar": "SPAM"},
    )


@ts.cli(Settings, "example")
def cli(settings: Settings) -> None:
    """Example app"""
    print(settings)


if __name__ == "__main__":
    cli()

Now compare the --help output with the example above:

$ python example.py --help
usage: example.py [-h] [--spam SPAM]

Example app

options:
  -h, --help   show this help message and exit

Settings:
  Settings options

  --spam SPAM  Spam count [default: 42]
example.py
import attrs
import typed_settings as ts


@attrs.frozen
class Settings:
    spam: int = attrs.field(
        default=42,
        metadata={
            "typed-settings": {
                "argparse": {
                    "help": "Spam count",
                    "metavar": "SPAM",
                },
            },
        },
    )


@ts.cli(Settings, "example")
def cli(settings: Settings) -> None:
    """Example app"""
    print(settings)


if __name__ == "__main__":
    cli()

Now compare the --help output with the example above:

$ python example.py --help
usage: example.py [-h] [--spam SPAM]

Example app

options:
  -h, --help   show this help message and exit

Settings:
  Settings options

  --spam SPAM  Spam count [default: 42]
example.py
import dataclasses

import typed_settings as ts


@dataclasses.dataclass
class Settings:
    spam: int = dataclasses.field(
        default=42,
        metadata={
            "typed-settings": {
                "argparse": {
                    "help": "Spam count",
                    "metavar": "SPAM",
                },
            },
        },
    )


@ts.cli(Settings, "example")
def cli(settings: Settings) -> None:
    """Example app"""
    print(settings)


if __name__ == "__main__":
    cli()

Now compare the --help output with the example above:

$ python example.py --help
usage: example.py [-h] [--spam SPAM]

Example app

options:
  -h, --help   show this help message and exit

Settings:
  Settings options

  --spam SPAM  Spam count [default: 42]
example.py
import pydantic
import typed_settings as ts


class Settings(pydantic.BaseModel):
    spam: int = pydantic.Field(
        default=42,
        # "description" will be copied into "typed_settings:argparse:help"
        description="Spam count",
        json_schema_extra={
            "typed-settings": {
                "argparse": {
                    "metavar": "SPAM",
                },
            },
        },
    )


@ts.cli(Settings, "example")
def cli(settings: Settings) -> None:
    """Example app"""
    print(repr(settings))


if __name__ == "__main__":
    cli()

Now compare the --help output with the example above:

$ python example.py --help
usage: example.py [-h] [--spam SPAM]

Example app

options:
  -h, --help   show this help message and exit

Settings:
  Settings options

  --spam SPAM  Spam count [default: 42]

Note

It is not possible to retrieve an option’s docstring directly within a Python program. Thus, Typed Settings can not automatically use it as help text for a command line option.

Since setting a help string is a very common use case, option() and secret() have a help argument as a shortcut to argparse={"help": "..."}.

Configuring Loaders and Converters

When you just pass an application name to cli() (as in the examples above), it uses default_loaders() to get the default loaders and default_converter() to get the default converter.

Instead of passing an app name, you can pass your own list of loaders:

import typed_settings as ts


@ts.settings
class Settings:
    spam: int = 42


loaders = [ts.loaders.EnvLoader(prefix="EXAMPLE_")]

@ts.cli(Settings, loaders)
def cli(settings: Settings):
    pass

In a similar fashion, you can use your own converter:

import typed_settings as ts


@ts.settings
class Settings:
    spam: int = 42


converter = ts.default_converter()
# converter.register_structure_hook(my_type, my_converter)


@ts.cli(Settings, "example", converter=converter)
def cli(settings: Settings):
    pass

Working with the ArgumentParser and Namespace

If you don’t like decorators or want to manually modify/extend the generated ArgumentParser, you can use the functions typed_settings.cli_argparse.make_parser() and typed_settings.cli_argparse.namespace2settings(). They can also be useful for testing purposes.

Here’s an example:

example.py
import typed_settings as ts


@ts.settings
class Settings:
    spam: int = ts.option(default=42, help="Spam count")


def main():
    # Create an argument parser with options for all settings:
    parser, merged_settings = ts.cli_argparse.make_parser(Settings, "example")
    print(parser)
    # You could now modify/extend the ArgumentParser

    # Parse the command line args (from "sys.argv"):
    namespace = parser.parse_args()
    print(namespace)

    # Convert the Namespace to an instance of your settings class:
    settings = ts.cli_argparse.namespace2settings(
        Settings, namespace, merged_settings=merged_settings,
    )
    print(settings)


if __name__ == "__main__":
    main()

$ python example.py --spam=3
ArgumentParser(prog='example.py', ...)
Namespace(spam=3)
Settings(spam=3)

Extending Supported Types

The type specific keyword arguments for argparse.ArgumentParser.add_argument() are generated by a thing called TypeArgsMaker. It is framework agnostic and uses a TypeHandler that actually generates the framework specific arguments for each type.

For argparse, this is the typed_settings.cli_argparse.ArgparseHandler. The easiest way to extend its capabilities is by passing a dict to it that maps types to specialized handler functions. The typed_settings.cli_argparse.DEFAULT_TYPES contain handlers for datetimes and enums.

Note

The ArgparseHandler supports so many common types that it was quite hard to come up with an example that makes at least some sense …;-)).

The following example is split into several smaller parts to make it easier to describe what’s going on. At the end, you’ll find the complete example in a single box.

Let’s assume you want to add support for a special dataclass that represents an RGB color and that you want to use a single command line option for it (like --color R G B).

import attrs
import dataclasses

@dataclasses.dataclass
class RGB:
    r: int = 0
    g: int = 0
    b: int = 0


@ts.settings
class Settings:
    color: RGB = RGB(0, 0, 0)

Note

If we used attrs instead of dataclasses here, Typed Settings would automatically generate three options --color-r, --color-g, and --color-b.

Since we want to create an instance of RGB from a tuple, we need to register a custom converter for it:

converter = ts.default_converter()
converter.register_structure_hook(
    RGB, lambda val, cls: val if isinstance(val, RGB) else cls(*val)
)

Next, we need to create a type handler function (see the TypeHandlerFunc protocol) for our dataclass. It must take a type, a default value and a flag that indicates whether the type was originally wrapped with typing.Optional. It must return a dictionary with keyword arguments for argparse.ArgumentParser.add_argument().

For our use case, we need an int option that takes exactly three arguments and has the metavar R G B. If (and only if) there is a default value for our option, we want to use it.

from typed_settings.cli_utils import Default, StrDict

def handle_rgb(_type: type, default: Default, is_optional: bool) -> StrDict:
    type_info = {
        "type": int,
        "nargs": 3,
        "metavar": ("R", "G", "B"),
    }
    if default:
        type_info["default"] = dataclasses.astuple(default)
    elif is_optional:
        type_info["default"] = None
    return type_info

We can now create a ArgparseHandler and configure it with a dict of our type handlers.

type_dict = {
    **ts.cli_argparse.DEFAULT_TYPES,
    RGB: handle_rgb,
}
type_handler = ts.cli_argparse.ArgparseHandler(type_dict)

Finally, we pass that handler to TypeArgsMaker and this in turn to cli():

@ts.cli(
    Settings,
    "example",
    converter=converter,
    type_args_maker=ts.cli_utils.TypeArgsMaker(type_handler),
)
def cli(settings: Settings):
    print(settings)

Full example

example.py
import dataclasses

import attrs
import typed_settings as ts
from typed_settings.cli_utils import Default, StrDict


@dataclasses.dataclass
class RGB:
    r: int = 0
    g: int = 0
    b: int = 0


@ts.settings
class Settings:
    color: RGB = RGB(0, 0, 0)


converter = ts.default_converter()
converter.register_structure_hook(
    RGB, lambda val, cls: val if isinstance(val, RGB) else cls(*val)
)

def handle_rgb(_type: type, default: Default, is_optional: bool) -> StrDict:
    type_info = {
        "type": int,
        "nargs": 3,
        "metavar": ("R", "G", "B"),
    }
    if default:
        type_info["default"] = dataclasses.astuple(default)
    elif is_optional:
        type_info["default"] = None
    return type_info


type_dict = {
    **ts.cli_argparse.DEFAULT_TYPES,
    RGB: handle_rgb,
}
type_handler = ts.cli_argparse.ArgparseHandler(type_dict)


@ts.cli(
    Settings,
    "example",
    converter=converter,
    type_args_maker=ts.cli_utils.TypeArgsMaker(type_handler),
)
def cli(settings: Settings):
    print(settings)


if __name__ == "__main__":
    cli()

$ # Check if our metavar and default value is used:
$ python example.py --help
usage: example.py [-h] [--color R G B]

options:
  -h, --help     show this help message and exit

Settings:
  Settings options

  --color R G B  [default: (0, 0, 0)]
$ # Try passing our own color:
$ python example.py --color 23 42 7
Settings(color=RGB(r=23, g=42, b=7))

This sounds a bit involved and it is in fact a bit involved, but this mechanism gives you the freedom to modify all behavior to your needs.

If adding a simple type handler is not enough, you can extend the ArgparseHandler (or create a new one) and – if that is not enough – event the TypeArgsMaker.