For Developers‎ > ‎

Jinja

Jinja is a template engine for Python, which is used in a number of places in Chromium, particularly for generating bindings. It's also referred to as Jinja2, as it's on its second major version. Usage is generally easy, but it's a new macro language to learn, and a few tips can help.

Warning: make sure to list Jinja template files as inputs in the appropriate .gyp or .gypi file, just like the Python files that use them – they are source code – otherwise the generated files will not rebuild properly.

Notable uses:

Overview


Jinja templates provide both basic string substitution (fill in the blank, as in Python template strings) and more advanced programming facilities. Key features are as follows:
  • if (conditional): {% if %}...{% elif %}...{% else %}...{% endif %}
  • for (definite iteration): {% for i in l if i.foo %}...{% endfor %}
Note that linebreaks are not meaningful in Jinja block control statements, so you can put linebreaks wherever convenient, and don't need to use () for implicit line continuation.
  • Macros: Macros are actually functions (evaluation semantics, not expansion semantics: they are evaluated as a function, not expanded in place like a macro). However, they can be called with the current evaluation environment ("context"), which results in dynamic binding, with results like actual macros; see Import Context Behavior. Macros are suitable for producing text, but filters are clearer for transforming text, notably simple inline transforms (like changing capitalization) or wrapping or indenting a block of text.
  • Blocks: Blocks are a bit trickier to use than macros, and are generally used like actual macros (by default calling environment is visible). They are powerful in general use, letting you have multiple templates based off of a base template, but in code generators, they are mostly used when you want to optionally include or omit a block of code, and don't want to specify arguments (not a parametrized function, just a block that uses the current environment). If you always want to include code, a macro is easier, and if you want to specify arguments, a macro is clearer.
Others:
  • set (assignment): {% set x = '' %} particularly useful with conditional expression: {% set x = 'foo' if foo else 'bar' %}
In rare cases call is useful, but usually a custom filter works and is clearer.

Note that Jinja identifiers are lowercase, and thus the 3 literal constants are written true, false, none, unlike in Python.

Factoring code

Macros can be put into a library and then imported into the template that uses them. This should generally be done (even for simple macros or those used in only one file), so that the code (macros) are separate from the data (template), and the actual template looks like the output, without being cluttered with macro definitions. This also allows you to have whitespace between macro definitions in the library.

To put macros into a library, just put them into another file and import them with an import statement. In terms of name binding, you often want macros to access the global environment – so you don't need to pass everything explicitly as an argument, and to simplify factoring code into a macro – which you can do via the with context keywords,

Factoring code using blocks is a bit trickier, since you need to use template Inheritance, specifically one base template with the block layout, and a derived template that fills in values for the block. See interface_base.cpp and interface.cpp in the V8 bindings code generator for example of using blocks and template inheritance.

Filters

Filters are very useful for transforming text functionally in Jinja, and can simplify code significantly. There are many builtin filters, often with powerful features, and you can write custom filters. In terms of use, you can either apply filters to variables, either in an expression expansion {{ ... }} or in a block control statement {% ... %}, or create a filter section to apply a filter to a portion of the template via {% filter ... %}...{% endfilter %}.

Particularly useful filters for expression expansion include:
  • indent: indent, indent(8), indent(8, true) – very useful for nested code via multi-line macros; see above.
  • join: join(', ') – very useful for parameter lists. This can also take an attribute name, so join(', ', 'name') will join the name attributes of the elements of the sequence.
More rarely useful ones include:
Particularly useful filters for block control statements include:
  • dictsort: {% for i in d|sort %}, {% for i in d|sort(false, 'name') %}
  • length: {% if foo|length > 1 %}
  • sort: {% for x in l|sort %}, {% for x in l|sort(attribute='name') %} – note that given a Python set, it is clearer to convert it to a sorted list via l = sorted(s) in Python, but for unsorted lists, it's clearer to sort in Jinja.

Jinja environment

The overall setup of Jinja calls the jinja2.Environment constructor. There are a few flags which are very useful for whitespace control, and should be used (these need to be manually specified due to backwards compatibility); these are indicated below. After initializing the environment, if you have any custom filters, you'll need to add them to environment.filters.
jinja_env = jinja2.Environment(
    loader=jinja2.FileSystemLoader(templates_dir),
    keep_trailing_newline=True,  # newline-terminate generated files
    lstrip_blocks=True,  # so can indent control flow tags
    trim_blocks=True)  # so don't need {%- -%} everywhere
jinja_env.filters.update({
    'foo': foo,
    # ...
})

Style

Python/Jinja split

In Jinja usage, the logic and code generation is not strictly split between the Python logic and the Jinja templates, and the boundary is sometimes a matter of taste.

To understand a Jinja template, please open both the Jinja template and the Python script that computes the context and refer to both, preferably by mainly reading the Jinja, and referring to the Python when necessary to understand how variables are computed.

As a rule:
  • multi-line code should be in templates,
  • components should be assembled in templates, and
  • complex logic should be in Python.
Simple logic (list filtering, ternary operations, simple auxiliary variables) should be in templates, but complex logic (anything that doesn't fit in a single expression), should be in Python, and the Jinja side should just have a variable.

As an example of the boundary is a method call. Unless there is complex logic, this should be assembled primarily in the template:
{{method.namespace}}::{{method.name}}({{method.arguments | join(', ')}});

By contrast, if there is complex logic, the logic should be in Python, and the template code should just contain a variable:
{{method.call_expression}};
Note that semicolons and linebreaks should be in the template, not the Python code!

For lists: simple iteration and filtering should be in Jinja, but list building or complex iteration should generally go in Python.

A good example of complex logic in the bindings generator is the expression for a Blink getter (method to call and arguments to pass), which is computed by getter_expression() in Python, and used as {{attribute.cpp_value}} in Jinja (so called because it might be 'resultValue' instead, in cases where the value is stored in a local variable in the getter method).

Do not call Python functions from Jinja (except custom filters)

Python functions should be called in Python to generate context values; this ensures a one-directional Python → Jinja pipeline. It is possible to pass Python functions as Jinja context values, and then call them in Jinja, but this makes the template hard to read and introduces a Python ⇄ Jinja cycle, which makes it harder to understand the code. This does create distance between computation and use (e.g., if there's a variable name in the template, and you want to compute a function call using that variable): if a Jinja context value should be computed in a certain way, adding a brief comment to that effect in the template is fine.

Note that custom filters are Python functions, but they are intended to be called from Jinja, and are idiomatic.

Python

One dictionary display per context, inline expressions

Ideally, the Python logic for a given context will be a single dictionary display (like a dictionary literal, but keys and values can be expressions, not literals), with the keys being (string) literals and the values being Python expressions: the function is just return {'bar': ..., 'foo': ..., ...}. This means you can just look up the key in the context-generation code, and is very functional and flat (instead of building up the context in various functions, which makes lookup harder). In the bindings generator, v8_methodsmethod_context and argument_context (currently generate_method and generate_argument) are good examples: each corresponds to a single object type (IdlOperation and IdlArgument), and generates a dict, nested corresponding to the nesting of objects.

Logic and local variables before display

However, in some cases the code needs to do additional processing, not just produce a context value. In that case the additional logic should come before the display, and use local variables, which are then used as values in the dictionary display. For example, if there are side effects, this should come before the display: compute a local variable, test that variable and have side effects if necessary, then use the variable in the display (no local variable needed if value not needed in context, only side effects). A more complex case is when one context value is a combination of several others; in that case the individual values are computed before the display, stored in local variables, and then combined in the display.

Separate functions if necessary

In some cases complex calculations are better factored into a separate function, with the main dictionary updated via dict.update(). If later calculations depend on earlier ones, the context computed so far can be passed in as an argument, and the dict updated in place. In the bindings generator a good example is the context for getters and setters for attributes: the logic is relative complicated, and depends on previously computed variables, so this is factored into separate functions, and the existing context dict is passed in as an argument, which is updated in place.

Early return if necessary

In rare cases, the context-generation code will have an early return after the main display, and then additional processing afterwards.

Jinja

Keep it simple

Jinja templates are in a domain-specific language (DSL), which is not familiar to most developers. Thus try to keep it simple.

Spacing

We use the following spacing:
{{foo}}
{{foo | filter}}
{% for i in x|filter %}
I.e., no spacing within {{}}, but do space around |, except in block control statements. This is the opposite of Jinja spec convention, which is {{ foo }} and {{ foo|filter }} – reasoning is that {{}} are functioning like parentheses or quotes, hence no spacing inside them; while | is functioning like a binary operator, like arithmetic operations, hence spaces around it. However, in block control statements | has higher precedence than the control flow keywords, and thus omitting spaces makes the precedence clearer.
More pragmatically, {{}} is very widely used for variable substitution, hence want to keep it short and looking like a single unit, while usage of filters via | is logically complex, and thus should stick out visually, with the steps in the pipeline clearly separated. However, adding spaces within a block control statement makes it confusing, due to the spaces around the control flow keywords, and thus spaces around | should be omitted.

set if simple and used in only one place

It is ok to use a set statement if a value is only useful in one place, and can be derived from existing data, which avoids cluttering the context display, but if a value is used multiple places or would require additional auxiliary variables just for it, it's better to compute it in the context display.

Variables should be complete words, not fragments

Variables should be complete words, not fragments, notably not affixes (prefixes and suffixes).
This is particularly important for literal values, due to searching, but also goes for variable values.
Complete words are more legible, and can be searched for automatically.
For example:
{% set ref_ptr = 'RefPtrOrNull' if nullable else 'RefPtr' %}
...
...{{ref_ptr}}...

Compare to the use of fragments (don't do this!):
{% set or_null = 'OrNull' if nullable else '' %}
...
...RefPtr{{or_null}}... // try grepping for "RefPtrOrNull"!

This also goes if the root itself is a variable: modify the variable via set instead of creating a new affix variable.
This keeps the variable as a unit, and makes the substitution easier to read:
{% set bar = bar + ('Constructor' if is_constructor else '') %}
..., {{foo}}, {{bar}}, ...

Compare to the use of fragments (don't do this!):
{% set bar_suffix = 'Constructor' if is_constructor else '' %}
..., {{foo}}, {{bar}}{{bar_suffix}}, ...

Name variables after default value

A common use case is for a variable to have a default value, but sometimes have a different value, particularly an affix. This is easier to read if you name the variable after the default value, like:
'foo': 'foo.bar' if bar else 'foo',
or:
{% set foo = 'foo.bar' if bar else 'foo' %}
...which can then be used as {{foo}} in templates, which mentally translates as "usually foo, but sometimes something else".

Comment end of long blocks

If a block is long, particularly if it is nested, please add a comment at the end to help clarify which block is ending; use the same content as the if condition or for list.
{% if foo %}
...
{% if bar == 'zork' %}
...
{% endif %}{# bar == 'zork' #}
...
{% endif %}{# foo #}
...
{% for x in l %}
...
{% endfor %}{# x in l #}

Note that if there are nested blocks, there's no need to comment the end of a short inner block, but you should comment the outer block:
{% if foo %}
{% if bar %}
...
{% else %}
...
{% endif %}
{% else %}{# foo #}
{% if bar %}
...
{% else %}
...
{% endif %}
{% endif %}{# foo #}

Use implicit False

As in usual Python style (True/False evaluations), you can use implicit FalseContext variables can be used as both content (filled into expressions) and control (determining control flow); a common technique is to check for the existence of a content variable as a conditional, which avoids a separate boolean variable just for control. Compare {% if foo %}...{{foo}}...{% endif %} to {% if is_foo %}...{{foo}}...{% endif %}.

Whitespace

Whitespace handling can be very fiddly, particularly around line breaks.

First, make sure you've set up your Jinja environment with the whitespace control flags – keep_trailing_newline, lstrip_blocks, trim_blocks – this will ensure that whitespace will be generally sane, and you can use blocks without needing to worry about whitespace most of the time. In old code you'll often see lots of explicit {%- ... -%} which are not necessary with these flags.

Secondly, it's easiest to define macros in a separate library, as this allows you to not worry about the whitespace caused by the macro definitions themselves, and also yields better factoring.

The default behavior these flags give is to treat lines that only contain a block (possibly indented) as not there whitespace-wise. Thus:
    {% if foo %}
    ...
    {% endif %}
...will only output a single line, despite the 2 lines of (indented) block statements.

Note however that expression evaluation in {{ ... }} are not trimmed; this is a particular issue for macros. Thus:
{{foo()}}
...does include a trailing newline, even if foo() evaluates to empty (''). Thus if you would like a section of the template to be omitted entirely in some cases, it is simplest to use a block, as in template inheritance.

Manual whitespace handling can be specified with {%-, -%}, {{-, -}}, to trim surrounding whitespace, and {%+ to not strip a leading indent. Trimming can be very confusing, so it's generally best to only use it when there's a literal boundary (literal text that will be stripped until), not when next to another block or expression; this also means to strip at the innermost level.

Cases where manual whitespace handling commonly occur:
  • In-line macros: end with {%- endmacro %}, or strip inside conditional block as {%- else %}...{%- endif %}
For a macro that's expected to be used inline  like a, {{f()}}, b – you need to trim or omit the trailing newlines in the macro definition, otherwise it will include a newline when expanded. This can be done via {%- as follows. Note that you only need to trim immediately after the newline, not repeatedly.
{% macro f() %}
...
{%- endmacro %}

{% macro g() %}
...
{%- else %}
...
{%- endif %}
{% endmacro %}

These are equivalent to omitting the line breaks, but allow more legible block positioning; omitting linebreaks yields the less legible:
{% macro foo() %}
...{% endmacro %}

{% macro g() %}
...{% else %}
...{% endif %}
{% endmacro %}
  • Block at start of line, keep indent: use {%+
If you need to include an optional start of an indented line, use {%+ to keep the initial indent:
    ...
    {%+ if foo %}... {% endif %}...
    ...
  • Split single line across multiple lines: avoid, use {%- ... -%} (- next to excess newline)
It's best to avoid complex conditional logic within a line. It's often simpler to set an auxiliary variable before the line and then just include that. Similarly, given multiple conditionals, such as an arguments list, it's often easier to have a list variable for the arguments and then build the arguments with {{args | join(', ')}} as in:
{% set args = ['foo'] if foo else [] %}
{% set args = args + ['bar'] %}
{% set args = args + (['zork'] if zork else []) %}
f({{args | join(', ')}});

However, if you wish to include the logic inline and split across multiple lines for legibility, the rule is: "use - next to the excess newline", making sure you're ultimately surrounded by literal text. For one-line conditionals, you just need {%- and %-} for the outermost tags. A common use is argument lists; note the spacing and comma usage.
f(
    {%- if foo %}foo, {% endif -%}
    bar
    {%- if zork %}, zork{% endif -%}
);

The actual whitespace rules mean that you can omit some of these trim instructions (if there are consecutive blocks, for instance), but it is simpler and more robust to include them (this lets blocks be added and removed without worrying about whitespace, for instance). For extra-long lines where you want to split the condition and contents across separate lines, you should use {%- ... -%} for each block, to ensure trimming:
f(
{%- if very_long_condition_indeed -%}
very_long_argument_name_with_bells_and_whistles
{%- else -%}
other_very_long_argument_name_with_bells_whistles_and_ribbons_too
{%- endif -%}
);

  • Long block at start of line, keep indent and split: use {%+ ... -%}
The above two can be combined if you have a long block at the start of a line. This is the most complex case you are likely to encounter:
    {%+ if foo %}[VeryLongExtendedAttributeName, AndThenAnotherOne] {% endif -%}
    attribute DOMString name;

Treat multi-line macros as blocks (indent, trailing semicolon ; in macro)

If a macro expands to multiple lines of output in an indented block (common in generating C++ code), in order for the output to be properly indented:
  • Do not indent the body of the macro.
  • Use the indent filter at the call site.
The indent defaults to adding an indent to all but the first line; use indent(8) (or indent(12) etc.) if you need to indent further, and indentfirst=True if you need to indent the first line as well, as indent(8, True) or indent(indentfirst=True). For more complex logic, like not indenting preprocessor directives in C++, you'll need to write a custom filter.

For example, definition:
{% macro f(x) %}
{% if x %}
a(x);
b(x);
{% else %}
a(y);
b(y);
{% endif %}
{% endmacro %}

Use:
    ...
    b();
    {{f() | indent}}
    c();
    ...

Treat one-line macros as in-line (no indent, {%- to strip trailing linebreak, semicolon ; at call site)

Some macros generate exactly one line of content. These should be treated as in-line macros: they do not need an indent, you should strip the trailing linebreak in the macro, and any trailing context (notably a semicolon) should be at the call site. This simplifies both the macro and the call, improves consistency, and emphasizes that it is a one-line macro.

For example, definition:
{% macro f(x) %}
{% if x %}
a(x)
{%- else %}
a(y)
{%- endif %}
{% endmacro %}

Use:
    ...
    b();
    {{f()}};
    c();
    ...

C++ code generation

Templated code generation (esp. C++ code) is particularly complicated to read, since it requires following both the flow of the template and of the generated code.

Don't mix Jinja templates and C++ templates

Jinja templates for generating C++ code, and C++ templates (in the usual sense) have the same effect: they are transformed into C++ code, which can then be compiled. These thus fill similar roles, and one can in principle use either for some purposes. If a C++ template is only being used within Jinja-generated code, it is clearer to replace it with a Jinja template, so you don't need to keep track of 2 template systems at once.
It is fine to use existing C++ templates that are used outside of the Jinja-generated code.

Group related code, minimize code within clauses

As much as possible, group related code together, particularly by having variable definitions close to use, especially immediately before; and minimize code within template clauses, particularly by having more generic code and using auxiliary variables.

For example:
{# Bad #}
T x = ...;
...
...
{% if c %}
f(x, g(...));
{% else %}
T y = h(...);
f(x, y);
{% endif %}

{# Good #}
T x = ...;
{% if c %}
T y = g(...);
{% else %}
T y = h(...);
{% endif %}
f(x, y);

Tips

Syntax highlighting

Syntax highlighting for Jinja is tricky, because ideally one wants highlighting for both languages: the target language (e.g., C++, HTML) and Jinja. Indeed, ideally one would have separate highlight colors for the target language and for Jinja (e.g., bold, inverse, or different background). This can be done, with some care.

For simple editing, it's fine to just use the highlighting of the target language (this is a key reason to use the extensions of the underlying file type for templates) and deal with Jinja code not being highlighted or appearing broken, but if you edit templates frequently, proper syntax highlighting is very helpful.

vim

In Vim, one can achieve nested syntax highlighting of Jinja via the following 4 steps:
  1. Download jinja.syntax file, place in ~/.vim/syntax
    (from 
    Jinja : adds jinja highlighting support to vim, link is to v1.1; you might want to make one minor update, but otherwise do not need to edit this file)
  2. Add nested highlighting function to ~/.vimrc
    (per below; 
    follows “Different syntax highlighting within regions of a file”, with some modifications)
  3. Add auto commands to call this function to ~/.vimrc
    (depends on filename convention; sample below)
  4. (optional) Add separate Jinja-specific highlighting groups.
" Per:
" Different syntax highlighting within regions of a file
" http://vim.wikia.com/wiki/Different_syntax_highlighting_within_regions_of_a_file
" Important changes:
" * Add keepend, otherwise nested C++/Jinja doesn't work!
" * Add containedin=ALL, so also highlighted in C comments and strings.
" * Remove the textSnipHl section (since want to include the delimiters
"   for Jinja).
"
" ...and using syntax from:
" http://www.vim.org/scripts/script.php?script_id=1856

function! TextEnableCodeSnip(filetype,start,end) abort
  let ft=toupper(a:filetype)
  let group='textGroup'.ft
  if exists('b:current_syntax')
    let s:current_syntax=b:current_syntax
    " Remove current syntax definition, as some syntax files (e.g. cpp.vim)
    " do nothing if b:current_syntax is defined.
    unlet b:current_syntax
  endif
  execute 'syntax include @'.group.' syntax/'.a:filetype.'.vim'
  try
    execute 'syntax include @'.group.' after/syntax/'.a:filetype.'.vim'
  catch
  endtry
  if exists('s:current_syntax')
    let b:current_syntax=s:current_syntax
  else
    unlet b:current_syntax
  endif
  execute 'syntax region textSnip'.ft.'
  \ start="'.a:start.'" end="'.a:end.'"
  \ keepend
  \ containedin=ALL
  \ contains=@'.group
endfunction

To automatically highlight Jinja files, add auto commands to turn on the nested highlighting. The following assumes that files in a directory named templates are Jinja templates:
" Jinja template highlighting
" Default delimiters are {{ }}, {% %}, and {# #}, per:
" http://jinja.pocoo.org/docs/templates/
au BufNewFile,BufRead */templates/* call TextEnableCodeSnip('jinja', '{{', '}}')
au BufNewFile,BufRead */templates/* call TextEnableCodeSnip('jinja', '{%', '%}')
au BufNewFile,BufRead */templates/* call TextEnableCodeSnip('jinja', '{#', '#}')

Jinja 2.7 slightly changed the syntax, adding {%+ ... %}, so the syntax file needs a slight change (add +\? two places, following -\? – beware of vim's idiosyncratic regex syntax). The following patch applies this:
--- 1.1/jinja.vim
+++ 1.2/jinja.vim
@@ -58 +58 @@
-syn region jinjaTagBlock matchgroup=jinjaTagDelim start=/{%-\?/ end=/-\?%}/ skipwhite containedin=ALLBUT,jinjaTagBlock,jinjaVarBlock,jinjaRaw,jinjaString,jinjaNested,jinjaComment
+syn region jinjaTagBlock matchgroup=jinjaTagDelim start=/{%-\?+\?/ end=/-\?%}/ skipwhite containedin=ALLBUT,jinjaTagBlock,jinjaVarBlock,jinjaRaw,jinjaString,jinjaNested,jinjaComment
@@ -71 +71 @@
-syn match jinjaStatement containedin=jinjaTagBlock contained skipwhite /\({%-\?\s*\)\@<=\<[a-zA-Z_][a-zA-Z0-9_]*\>\(\s*[,=]\)\@!/
+syn match jinjaStatement containedin=jinjaTagBlock contained skipwhite /\({%-\?+\?\s*\)\@<=\<[a-zA-Z_][a-zA-Z0-9_]*\>\(\s*[,=]\)\@!/

You can apply this via cd ~/.vim/syntax && patch -p1 by just copy-pasting (hit ^D^D to terminate), or use a temporary file as in:
cd ~/.vim/syntax && patch -p1 < /tmp/jinja.vim.1_1-1_2.diff

Lastly, if you wish to have separate highlighting for Jinja (e.g., have C++ comments and Jinja comments highlighted differently), add highlight commands for Jinja-specific syntax groups (listed in jinja.vim, particularly the last block), such as jinjaComment. Note that these must be specified manually (not just "copy existing group and change one trait"), since Vim syntax highlighting does not support such inheritance.

Emacs

jinja2-mode is a major mode for Jinja; web-mode may also work.
See Multiple Modes for libraries providing multiple major modes in a single buffer.
FIXME: do these work, say for C++ code?

Performance

http://jinja.pocoo.org/docs/api/#bytecode-cache
https://groups.google.com/forum/#!topic/pocoo-libs/dIsNICT4j2Q
https://gist.github.com/voscausa/9055838
http://jinja.pocoo.org/docs/api/#jinja2.Environment.compile_templates

External links


Comments