Internationalization and localization tags and filters

docs/.overrides/main.html

@@ -0,0 +1,5 @@
+{% extends "base.html" %}
+
+{% block announce %}
+  This documentation is under construction.
+{% endblock %}

docs/api/babel.md

@@ -0,0 +1,10 @@
+These are configurable class-based filters implementing internationalization and localization features.
+
+::: liquid2.builtin.filters.translate.BaseTranslateFilter
+::: liquid2.builtin.DateTime
+::: liquid2.builtin.GetText
+::: liquid2.builtin.NGetText
+::: liquid2.builtin.NPGetText
+::: liquid2.builtin.Number
+::: liquid2.builtin.Translate
+::: liquid2.builtin.Unit

docs/api/convenience.md

@@ -1,3 +1,4 @@
 ::: liquid2.parse
 ::: liquid2.render
 ::: liquid2.render_async
+::: liquid2.extract_liquid

docs/migration.md

@@ -29,6 +29,8 @@ The following features are new or are now built-in where they weren't before.
 - Ternary expressions are now available by default. For example, `{{ a if b else c }}` or `{{ a | upcase if b == 'foo' else c || split }}`.
 - Inline comments surrounded by `{#` and `#}` are enabled by default. Additional `#`’s can be added to comment out blocks of markup that already contain comments, as long as the number of hashes match.
 - String literals are allowed to contain markup delimiters (`{{`, `}}`, `{%`, `%}`, `{#` and `#}`) and support c-like escape sequence to allow for including quote characters.
+- Identifiers and paths resolving to variables can contain Unicode characters.
+- Integer and float literals can use scientific notation.
 - Filter and tag named arguments can be separated by a `:` or `=`.
 - Template inheritance is now built-in. Previously `{% extends %}` and `{% block %}` tags were available from a separate package.
 - Internationalization and localization tags and filters are now built-in. Previously these were in a separate package.
@@ -43,6 +45,8 @@ These features are not yet included in Python Liquid2, but can be if there is a
 - Async filters have not been implemented.
 - Contextual template analysis has not been implemented.
 - Template tag analysis (analyzing tokens instead of a syntax tree) has not been implemented.
+- The `@liquid_filter` decorator has been removed. Now filter implementations are expected to raise a `LiquidTypeError` in the even of an argument with an unacceptable type.
+- Liquid Babel used to allow simple, zero-argument filters in the arguments to the `translate` tag. The `translate` tag bundled in to Liquid2 does not allow the use of filters here.
 
 ## API changes
 
@@ -55,6 +59,7 @@ These are the most notable changes. Please raise an [issue](https://github.com/j
 - The `auto_reload` and `cache_size` arguments to `Environment` have been removed. Now caching is handle by template loaders, not the environment. For example, pass a `CachingFileSystemLoader` as the `loader` argument to `Environment` instead of a `FileSystemLoader`.
 - `TemplateNotFound` has been renamed to `TemplateNotFoundError`.
 - `Context` has been renamed to `RenderContext` and now takes a mandatory `template` argument instead of `env`. All other arguments to `RenderContext` are now keyword only.
+- `FilterValueError` and `FilterArgumentError` have been removed. `LiquidValueError` and `LiquidTypeError` should be used instead. In some cases where `FilterValueError` was deliberately ignored before, `LiquidValueError` is now raised.
 
 ### Template and expression parsing
 
@@ -79,3 +84,4 @@ The following packages are dependencies of Python Liquid2.
 - Markupsafe>=2
 - Babel>=2
 - python-dateutil
+- pytz

liquid2/__init__.py

@@ -1,5 +1,7 @@
 from typing import Any
+from typing import Iterator
 from typing import Mapping
+from typing import TextIO
 
 from .token import BlockCommentToken
 from .token import CommentToken
@@ -45,23 +47,31 @@
 from .undefined import StrictUndefined
 from .undefined import Undefined
 from .exceptions import TemplateNotFoundError
+from .messages import MessageTuple
+from .messages import extract_from_template
 
 from .__about__ import __version__
 
 DEFAULT_ENVIRONMENT = Environment()
 
 
-def parse(source: str, globals: Mapping[str, object] | None = None) -> Template:
+def parse(
+    source: str,
+    *,
+    name: str = "",
+    globals: Mapping[str, object] | None = None,
+) -> Template:
     """Parse _source_ as a Liquid template using the default environment.
 
     Args:
         source: Liquid template source code.
+        name: An optional name for the template used in error messages.
         globals: Variables that will be available to the resulting template.
 
     Return:
         A new template bound to the default environment.
     """
-    return DEFAULT_ENVIRONMENT.from_string(source, globals=globals)
+    return DEFAULT_ENVIRONMENT.from_string(source, name=name, globals=globals)
 
 
 def render(source: str, *args: Any, **kwargs: Any) -> str:
@@ -99,6 +109,39 @@ async def render_async(source: str, *args: Any, **kwargs: Any) -> str:
     return await template.render_async(*args, **kwargs)
 
 
+def extract_liquid(
+    fileobj: TextIO,
+    keywords: list[str],
+    comment_tags: list[str] | None = None,
+    options: dict[object, object] | None = None,  # noqa: ARG001
+) -> Iterator[MessageTuple]:
+    """A babel compatible translation message extraction method for Liquid templates.
+
+    See https://babel.pocoo.org/en/latest/messages.html
+
+    Keywords are the names of Liquid filters or tags operating on translatable
+    strings. For a filter to contribute to message extraction, it must also
+    appear as a child of a `FilteredExpression` and be a `TranslatableFilter`.
+    Similarly, tags must produce a node that is a `TranslatableTag`.
+
+    Where a Liquid comment contains a prefix in `comment_tags`, the comment
+    will be attached to the translatable filter or tag immediately following
+    the comment. Python Liquid's non-standard shorthand comments are not
+    supported.
+
+    Options are arguments passed to the `liquid.Template` constructor with the
+    contents of `fileobj` as the template's source. Use `extract_from_template`
+    to extract messages from an existing template bound to an existing
+    environment.
+    """
+    template = parse(fileobj.read())
+    return extract_from_template(
+        template=template,
+        keywords=keywords,
+        comment_tags=comment_tags,
+    )
+
+
 __all__ = (
     "__version__",
     "BlockCommentToken",
@@ -133,6 +176,8 @@ async def render_async(source: str, *args: Any, **kwargs: Any) -> str:
     "PathT",
     "PathToken",
     "RawToken",
+    "render_async",
+    "render",
     "RenderContext",
     "StrictUndefined",
     "Tag",
@@ -147,4 +192,5 @@ async def render_async(source: str, *args: Any, **kwargs: Any) -> str:
     "Undefined",
     "unescape",
     "WhitespaceControl",
+    "extract_liquid",
 )

liquid2/builtin/__init__.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+from gettext import NullTranslations
 from typing import TYPE_CHECKING
 
 from .comment import Comment
@@ -50,6 +51,10 @@
 from .filters.array import sum_
 from .filters.array import uniq
 from .filters.array import where
+from .filters.babel import Currency
+from .filters.babel import DateTime
+from .filters.babel import Number
+from .filters.babel import Unit
 from .filters.math import abs_
 from .filters.math import at_least
 from .filters.math import at_most
@@ -90,6 +95,11 @@
 from .filters.string import upcase
 from .filters.string import url_decode
 from .filters.string import url_encode
+from .filters.translate import GetText
+from .filters.translate import NGetText
+from .filters.translate import NPGetText
+from .filters.translate import PGetText
+from .filters.translate import Translate
 from .loaders.caching_file_system_loader import CachingFileSystemLoader
 from .loaders.choice_loader import CachingChoiceLoader
 from .loaders.choice_loader import ChoiceLoader
@@ -115,10 +125,12 @@
 from .tags.liquid_tag import LiquidTag
 from .tags.raw_tag import RawTag
 from .tags.render_tag import RenderTag
+from .tags.translate_tag import TranslateTag
 from .tags.unless_tag import UnlessTag
 
 if TYPE_CHECKING:
     from ..environment import Environment  # noqa: TID252
+    from ..messages import Translations  # noqa: TID252
 
 
 __all__ = (
@@ -201,6 +213,17 @@
     "TrueLiteral",
     "UnlessTag",
     "parse_string_or_path",
+    "register_translation_filters",
+    "Currency",
+    "GetText",
+    "NGetText",
+    "NPGetText",
+    "PGetText",
+    "Translate",
+    "DataTime",
+    "Number",
+    "Unit",
+    "DateTime",
 )
 
 
@@ -262,6 +285,23 @@ def register_standard_tags_and_filters(env: Environment) -> None:  # noqa: PLR09
     env.filters["url_encode"] = url_encode
     env.filters["url_decode"] = url_decode
 
+    env.filters[GetText.name] = GetText(auto_escape_message=env.auto_escape)
+    env.filters[NGetText.name] = NGetText(auto_escape_message=env.auto_escape)
+    env.filters[NPGetText.name] = NPGetText(auto_escape_message=env.auto_escape)
+    env.filters[PGetText.name] = PGetText(auto_escape_message=env.auto_escape)
+    env.filters[Translate.name] = Translate(auto_escape_message=env.auto_escape)
+    env.filters["currency"] = Currency()
+    env.filters["money"] = Currency()
+    env.filters["money_with_currency"] = Currency(default_format="¤#,##0.00 ¤¤")
+    env.filters["money_without_currency"] = Currency(default_format="#,##0.00")
+    env.filters["money_without_trailing_zeros"] = Currency(
+        default_format="¤#,###",
+        currency_digits=False,
+    )
+    env.filters["datetime"] = DateTime()
+    env.filters["decimal"] = Number()
+    env.filters["unit"] = Unit()
+
     env.tags["__COMMENT"] = Comment(env)
     env.tags["__CONTENT"] = Content(env)
     env.tags["__OUTPUT"] = Output(env)
@@ -283,3 +323,50 @@ def register_standard_tags_and_filters(env: Environment) -> None:  # noqa: PLR09
     env.tags["__LINES"] = LiquidTag(env)
     env.tags["block"] = BlockTag(env)
     env.tags["extends"] = ExtendsTag(env)
+    env.tags["translate"] = TranslateTag(env)
+
+
+def register_translation_filters(
+    env: Environment,
+    *,
+    replace: bool = True,
+    translations_var: str = "translations",
+    default_translations: Translations | None = None,
+    message_interpolation: bool = True,
+    autoescape_message: bool = False,
+) -> None:
+    """Add gettext-style translation filters to a Liquid environment.
+
+    Args:
+        env: The liquid.Environment to add translation filters to.
+        replace: If True, existing filters with conflicting names will
+            be replaced. Defaults to False.
+        translations_var: The name of a render context variable that
+            resolves to a gettext `Translations` class. Defaults to
+            `"translations"`.
+        default_translations: A fallback translations class to use if
+            `translations_var` can not be resolves. Defaults to
+            `NullTranslations`.
+        message_interpolation: If `True` (default), perform printf-style
+            string interpolation on the translated message, using keyword arguments
+            passed to the filter function.
+        autoescape_message: If `True` and the current environment has
+            `autoescape` set to `True`, the filter's left value will be escaped
+            before translation. Defaults to `False`.
+    """
+    default_translations = default_translations or NullTranslations()
+    default_filters = (
+        Translate,
+        GetText,
+        NGetText,
+        PGetText,
+        NPGetText,
+    )
+    for _filter in default_filters:
+        if replace or _filter.name not in env.filters:
+            env.filters[_filter.name] = _filter(
+                translations_var=translations_var,
+                default_translations=default_translations,
+                message_interpolation=message_interpolation,
+                auto_escape_message=autoescape_message,
+            )