MiniJinja is a powerful but minimal dependency template engine for Rust

The current fields interface only allows borrowing a str out of self. While attempting to make a safe interface for stack borrowing I realized that there are cases where you really need to create an owned string there (#156).

There are two options here:

• fn fields(&self) -> Box<dyn Iterator<Item = Cow<'_, str>> + '_> — which has the advantage that you can borrow in a lot of cases, but it means that code which cannot hold on to the lifetime needs to allocate a string in all cases
• fn fields(&self) -> Box<dyn Iterator<Item = Cow<'static, str>> + '_> — would require more dynamic allocations in case the field name is not fully known, but has the advantage that we could internally start having a ValueRepr::StaticStr which might avoid unnecessary allocations in a lot of cases.

This changes the object interface to support both structs and sequences with the possibility to add extra behavior (eg: arbitrary key maps) at a future point. There are some challenges with this design however.

Issues:

These are probably something I need to accept:

• impl<'a, T: ArgType<'a, Output = T>> ArgType<'a> for Vec<T> cannot be implemented to support these new dynamic sequences because the lifetime constraints cannot be enabled. Restrict?

Things to resolve:

This is stuff that can be fixed:

• [x] Similarly the slice syntax x[:5] first converts the entire value before slicing down.
• [x] Decide on the names.
• [x] The Value::as_slice API cannot return borrowed values for dynamic slices. Internally a as_cow_slice has been added. Potential solution is to retire as_slice entirely for SeqObject.
• [x] Iteration over dynamic slices first makes a real slice out of it. This is mostly done because of lifetime issues that are annoying to work around.
• [x] Consider adding SeqObject::iter

Refs #147

My main use case for this is displaying recursive data. I think currently the only way to display recursive data with minijinja is by flattening it in rust code. Recursive include doesn't work either.

Would this feature still be in scope of minijinja? I'm not sure if the mini is because it doesn't support all features of jinja or because it doesn't plan to.

Would it be possible to add some sort of hook to allow my code to resolve the include files when they are needed, instead of adding them in ahead of time?

Right now, as far as I can tell, I have to know the names of all the files that are needed before-hand and add them to the environment, which requires my users to have to keep track of these and pass a list to me. I do see the Source struct which allows loading all templates in a path, but then the code would load all sorts of files that it doesn't need to load.

I am using this in a small tool to render configuration files, and want to be able to more easily translate existing knowledge of Jinja2 from Python to minijinja.

I am looking to do something like, which does not work:

{% for (k, v) in env.items() %}
{{ k }}: {{ v }}
{%- endfor %}

Error { kind: ImpossibleOperation, detail: Some("object has no method named items")

This does work:

{% for k in env %}
{{ k }}: {{ env[k] }}
{%- endfor %}

So I have a work-around for now.

I did see this closed issue: https://github.com/mitsuhiko/minijinja/issues/9 however there is no items filter.

Is there a way to add some of the more useful functions, such as keys(), items(), or values() on maps?

I missed a feature where i can ident a multiline Value. This PR should fix it.

Please be patient with me, this is my second PR and I'm new to Rust. I will do my best if something is missing.

Description

The 'filters::Filter' changed its API between these versions, you should increment the mayor.

Reproduction steps

This code works in 1.19.1

struct Test {}

impl minijinja::filters::Filter<String, String, ()> for Test
{
    fn apply_to(&self, _state: &minijinja::State, _value: String, _args: ()) -> Result<String, minijinja::Error>
    {
        Ok("".to_string())
    }
}

But fails in 1.20

error[E0053]: method `apply_to` has an incompatible type for trait
  --> src/filters.rs:24:81
   |
24 |     fn apply_to(&self, _state: &minijinja::State, _value: String, _args: ()) -> Result<String, minijinja::Error>
   |                                                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |                                                                                 |
   |                                                                                 expected struct `std::string::String`, found enum `Result`
   |                                                                                 help: change the output type to match the trait: `std::string::String`
   |
   = note: expected fn pointer `fn(&filters::Test, &minijinja::State<'_, '_>, std::string::String, ()) -> std::string::String`
              found fn pointer `fn(&filters::Test, &minijinja::State<'_, '_>, std::string::String, ()) -> Result<std::string::String, minijinja::Error>`

error: aborting due to 2 previous errors

Additional helpful information:

• Version of rustc: 1.63.0 (4b91a6ea7 2022-08-08)
• Operating system and version: Ubuntu 20.04

What did you expect

Should work because the mayor is the same (SemVer).

In Jinja, there is an extension called "expression statements":

https://jinja.palletsprojects.com/en/3.1.x/extensions/#expression-statement

The “do” aka expression-statement extension adds a simple do tag to the template engine that works like a variable expression but ignores the return value.

These are super handy, you can do things like this:

https://jinja.palletsprojects.com/en/3.1.x/templates/#expression-statement

Thoughts? (Happy to submit this)

I have a use-case where I'd like to keep track of a limited amount of global state across invocations of the same filter, for example, keeping track of the number of bound parameters to a sql query. Currently we can't get a mutable handle to the env, only &Environment. Initially I thought I could do this by passing a reference to a HashMap or some other shared data structure around into the filter, but it seems like the filter functions need to remain pure due to the automatic type conversion that is happening (which is great, btw).

jinjasql, the library which I'm trying to emulate handles this via thread local variables, is there an idiomatic way to share filter state across invocations?

Using Object cut down the running time of my application by 25% by allowing me to remove clones in favor of using proxy objects. From profiling, I can see that a similar possibility for optimization exists for sequences.

At present, if I have an existing sequence of non-minijinja-values, I have no recourse but to clone the entire sequence, converting each value into a native minijinja::Value in the process:

array.iter()
    .map(value_to_minijinja_value)
    .collect()

This is expensive, especially when the sequence is large. If instead there was a trait like Object for sequences, I could wrap the array in a proxy object like I do maps and save the cost:

Value::from_sequence_object(ProxySequence(array))

One option is to introduce another trait and ValueRepr kind, say Sequence, which looks a bit like Object:

pub trait Sequence: Display + Debug + Any + Sync + Send {
    fn get(&self, index: usize) -> Option<Value> { ... }

    /// This is a range so that slices are cheaply representable.
    fn range(&self) -> Range<usize> { ... }
}

Alternatively, Object can be generalized to be useful for sequence-like objects as well. One approach for this option is to add the same get() and range() methods above to Object. Another is to make the existing get_attr() and attributes() methods more versatile, allowing the former to take a usize and the latter to emit a range().

Consider the following (also here):

{% extends template %}
{% macro foo() %}inside foo{% endmacro %}
{% block title %}{{ foo() }}{% endblock %}
{% block body %}new body{% endblock %}

This gives the following error:

Error {
    kind: UnknownFunction,
    detail: "foo is unknown",
    name: "macro-extends.txt",
    line: 3,
}

unknown function: foo is unknown (in macro-extends.txt:3)
------------------------------ macro-extends.txt ------------------------------
   1 | {% extends template %}
   2 | {% macro foo() %}inside foo{% endmacro %}
   3 > {% block title %}{{ foo() }}{% endblock %}
     i                     ^^^^^ unknown function
   4 | {% block body %}new body{% endblock %}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
No referenced variables
-------------------------------------------------------------------------------

Removing the extends results in the template compiling without error.

I considered whether this was intended behavior, but the syntax docs make no mention of blocks having some newly defined scope. And if they do, then how would I go about invoking foo(), defined in a child template, inside of a block in said child?

Isolated from #157:

In the latter case, I have imagined an API by which the engine tracks object path accesses at compile-time and then asks once (say via an Object trait) for a terminal value that isn't compound, so anything but a map or sequence. For example, for foo.bar[0].baz, the engine would ask the foo object (assuming it is one) for bar[0].baz. The object can then perform an efficient lookup without any clones and return the terminal baz. Similarly, if we have:

{% set bar = foo.bar[0] %}
{{ bar.baz }}

The engine would again ask foo for bar[0].baz. This of course requires some thought and implementation grease, but, aside from allowing full borrows of anything, seems like the most sensible approach.

Today the engine holds &'source str all over the place in the instructions and by extension in the templates. This is a design pillar that I originally inherited from the initial API that was inspired by tinytemplate. It's a pretty clever setup but it has the disadvantage that the API cannot deal with dynamic templates well.

The solution to this problem so far has been the source feature which introduces a Source type which lets the engine lie about lifetimes through the use of self-cell and memo-map.

There is a alternative that could be evaluated where these borrows to the source string are not taking place through the references but string handles. That would likely require significant changes in the lexer and potentially the parser as well, but the changes to the instructions are probably quite limited.

The way this could work is that the instructions that currently hold an &'source str would instead hold something like a StrHandle which looks something like this:

struct StrHandle {
    offset: u32,
    len: u32,
    #[cfg(debug_assertions)]
    instructions_id: u64,
}

impl StrHandle {
    pub fn as_str(&self, instr: &Instructions) -> &str {
        debug_assert_eq!(instr.id, self.instructions_id);
        &instr.source[self.offset as usize..self.offset as usize + self.len as usize]
    }
}

Obviously one complexity with this approach is that at no point the handles must be separated from the instructions they are contained in. It also comes at least through the safe interface with additional cost as Rust will require a lot more bounds checks. unsafe would work here, but actually making a safe interface where someone does not change the source string underneath the instructions accidentally could be quite tricky.

The benefit of doing this would be that the engine ends up with less dependencies, overall less unsafe code and a nicer API.

There is quite a bit of ecosystem being built on top of valuable nowadays thanks to the tracing ecosystem. It might be worth exploring as an alternative/addition to serde.

Hi ! First of all thanks for this awesome lib !

Do you have any plans on implementing a way to provide custom delimiters ? Either at compile time or at runtime.

This feature does not exist in Tera, and Askama’s approach makes it impossi ble to have runtime provided templates. I am writing a CLI to parse and render jinja2 templates using [[ … ]], [% … %] and [# … #] to avoid conflicting with other templating languages used in the files I am processing.

Thanks in advance !

These two special loop variables are currently not supported but both very useful. loop.previtem points to the last value of the iteration, loop.nextitem to the next one.