Jello as Ansible filter

[g-v-egidy]

Hi,

I thought it would be helpful to have jello available as filter in Ansible.

Why you may ask?
I find the jmespath filter syntax that is available with community.general.json_query quite limited (for example in comparison to jc) and still as complicated as jc.

So I could create custom Ansible filter plugins instead. Yes, I've been doing that. But they have several downsides:

• Quite some boilerplate necessary for each filter
• The code has to be in a separate file and separate path (filter_plugins/). This makes reading the code flow in a playbook more difficult, you have to jump between the files
• No interactive debugging of complex data structures available, rerunning the playbook after each small code change can be slow.

I think jello would be a nice fit to help with that.

Here is a short task snippet how using it could look like:

- name: get human readable free disk space on local disks
  debug:
    msg: "{{ ansible_mounts | jello(jello_c) }}"
  vars:
  jello_c: |
        import humanize, re
        available = 0
        for mount in _:
        if re.match("^/dev",mount.device):
            available+=mount.size_available
        humanize.naturalsize(available)

Here is the code I used to implement this:
jello.py.txt

Do you think this is a good idea?

Does this implementation make sense to you or think it could be improved?

I'm thinking about two improvements:

1. Integrated support for jellex: I'm thinking about an option that automatically dumps the input and output data structures of jello to disk as json files. This would allow you to easily load them with jellex to work on the code. Unfortunately I haven't found a way yet to get proper filenames that represent the current task or step within the Ansible playbook.
2. Make the whole template environment available to the jello code. This would allow to loop over the inventory or similar things, without requiring to have the variables directly templated in the code. This would require an extension of the pyquery() function of jello that allows you to add things to the scope variable.

[kellyjonbrazil]

Hi there!

This is an interesting idea. I'm curious (and I haven't tested this) but since you can run a python interpreter within a playbook is there much difference? You would just need to import the json library and load the json string I think it would look something like this:

- name: filter json
  command: /usr/bin/python3
  args:
    stdin: |
      import json
      _ = json.loads({{ myjson }})
      print(_["mykey"])
  register: results

  vars:
      myjson: "{\"mykey\": \"myvalue\"}"

- set_fact:
    myfact: "{{ results.stdout }}"

[g-v-egidy]

Thanks for answering.

Yes, calling python like this would work, you don't even need json and can directly use the dict:

- name: filter json
  command: /usr/bin/python3
  args:
    stdin: |
      _ = {{ myjson }}
      print(_["mykey"])
  register: results

  vars:
      myjson: "{\"mykey\": \"myvalue\"}"

- set_fact:
    myfact: "{{ results.stdout }}"

But I think using jello has several advantages:

• no two tasks (command, set_fact) required. So it will execute faster.
• less boilerplate (for something I tend to use often, every additional line of boilerplate stuff counts in my opinion)
• it is a filter
  • can easily be chained with existing filters
  • can easily be reused multiple times
  • can be used in a template document

I think especially the last point is really helpful. Consider for example this where I need to regroup a complex data structure:

- name: write switch config
  template:
    src: switch-config.j2
    dest: /backup/new-config
  vars:
    vlan_filt: |
      vlans = { }
      for port, pcfg in _.items():
        for vlan in pcfg["vlans"]:
          if vlan not in vlans:
            vlans[vlan] = { }
            vlans[vlan]["untagged"] = set()
            vlans[vlan]["tagged"] = set()
          if vlan == pcfg["pvid"]:
            vlans[vlan]["untagged"].add(port)
          else:
            vlans[vlan]["tagged"].add(port)
      vlans

and the switch-conifg.j2 template then uses the code like this:

[...]
/interface bridge vlan
{% for vlan, vports in port_config | jello(vlan_filt) | dictsort %}
add bridge=bridge tagged={% for tp in vports.tagged %}{{ tp }},{% endfor %} untagged={% for utp in vports.untagged %}{{ utp }},{% endfor %} vlan-ids={{ vlan }}
{% endfor %}

You could of course also create a new datastructure with a call to python before and use that in the template. But I think using jello here is more elegant, shorter and faster.

[kellyjonbrazil]

Thank you for your input on this. I am subscribed to the Ansible development mailing list, which is the channel I used to develop the jc Ansible plugin. I'll start a thread there to see if this might be a fit within the community.general collection or somewhere else.

[g-v-egidy]

When you think this concept is a good idea, we could also refine it a bit before upstreaming:

I sometimes had issues with exceptions in my jello code. They weren't easy to pinpoint to the location in the code. But I didn't invest time to investigate this.

I suggest to adapt jello to allow customizing the scope that is used in the pyquery() function. This would allow the jello filter to get access to the whole template environment.

[kellyjonbrazil]

Nice idea - like a debug mode with -d or something where the scope is expanded to the rest of the environment. Maybe some other features when debug mode is enabled. I'll think about that. Thanks!