Upgrade to latest yarn version + upgrade all GH actions

[jibees]

Actually, do not close #838

• Yarn latest version
• Node no change
• GH Actions latest version each
• Python latest version
• Jinja2: lock to older version of Jinja2 (From memory, due to both Ansible 2.7 and our code)

Not sure how to test it actually. I guess this is a question for @mkllnk and @heroinedor ;)

[mkllnk]

Nice.

Not sure how to test it actually.

The first test is to use a local virtual machine to provision from scratch and see if the app works there. The next test is to use this branch to provision a staging server and make sure that nothing breaks.

[RachL]

@openfoodfoundation/core-devs anybody up for testing this? :)

[dacook]

Sorry I've been wanting to look into this and the questions about docker. Hopefully I can pick it up next week.

[dacook]

I spent time getting the vagrant VM working on my my environment, and when I ran setup.yml for a newly provisioned VM I got an error:

TASK [install yarn] ****************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
fatal: [local_vagrant]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 102, in <module>\n  File \"<stdin>\", line 94, in _ansiballz_main\n  File \"<stdin>\", line 40, in invoke_module\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_npm_payload_l51m5qhw/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 316, in <module>\n  File \"/tmp/ansible_npm_payload_l51m5qhw/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 293, in main\n  File \"/tmp/ansible_npm_payload_l51m5qhw/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 206, in list\n  File \"/usr/lib/python3.8/json/__init__.py\", line 357, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib/python3.8/json/decoder.py\", line 337, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib/python3.8/json/decoder.py\", line 355, in raw_decode\n    raise JSONDecodeError(\"Expecting value\", s, err.value) from None\njson.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

But I also seem to get an error when provisioning from the master branch, so might need to work on my environment some more. To be continued...

[jibees]

Maybe @heroinedor can take a look? I wonder if he's any closer to having such an environment ready.

[jibees]

From CI:

{
   "changed":false,
   "module_stderr":"Traceback (most recent call last):
File \"<stdin>\", line 102, in <module>
File \"<stdin>\", line 94, in _ansiballz_main
File \"<stdin>\", line 40, in invoke_module
File \"/usr/lib/python2.7/runpy.py\", line 188, in run_module
  fname, loader, pkg_name)
File \"/usr/lib/python2.7/runpy.py\", line 82, in _run_module_code
  mod_name, mod_fname, mod_loader, pkg_name)
File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code
  exec code in run_globals
File \"/tmp/ansible_npm_payload_Y_vIjC/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 316, in <module>
File \"/tmp/ansible_npm_payload_Y_vIjC/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 293, in main
File \"/tmp/ansible_npm_payload_Y_vIjC/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 206, in list
File \"/usr/lib/python2.7/json/__init__.py\", line 339, in loads
  return _default_decoder.decode(s)
File \"/usr/lib/python2.7/json/decoder.py\", line 364, in decode
  obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File \"/usr/lib/python2.7/json/decoder.py\", line 382, in raw_decode
  raise ValueError(\"No JSON object could be decoded\")\nValueError: No JSON object could be decoded\n",
   "module_stdout":"",
   "msg":"MODULE FAILURE\nSee stdout/stderr for the exact error",
   "rc":1
}

[sigmundpetersen]

Don't know if this is the issue, but there seems to be some discrepancies in how Python is setup.
From David's error message we can see Python 3.8 is used, while CI uses 2.7.
The version used should be consistent.

The version CI uses is configured here

ofn-install/.github/workflows/build.yml

Lines 31 to 35 in b5709e5

	- uses: actions/setup-python@v2
	with:
	python-version: '2.7'

The version used elsewhere I guess is from the readme https://github.com/openfoodfoundation/ofn-install#python

We have issues #663 and #765 regarding upgrading Python version, and I don't think we can close #663 just yet.

[jibees]

Build is green now with only the patch change on yarn_version.

When changing node version to version 16 or 18, it failed. Will try with 15.

[jibees]

Not working for node_version 15.x.

[jibees]

Upgrading to python 3.8.2 failed 😣

fatal: [local_test]: FAILED! => {"msg": "template error while templating string: No filter named 'regex_replace'.. String: nodejs={{ nodejs_version|regex_replace('x', '') }}*"}

Upgrading all GH actions is OK ✅

[sigmundpetersen]

I wonder if the Python fail is related to #842 and jinja2 dependency.
Maybe the best way is to upgrade ansible 🤷

[dacook]

Good pickup Sigmund. Yes I agree I think that latest error is specifically due to Jinja2 3.1. As mentioned on that issue, you could workaround by using an older version:

 pip install 'jinja2<3.1' --force-reinstall

[jibees]

Maybe the best way is to upgrade ansible 🤷

Welcome to the nightmare of dependencies when a project is not up to date ;)

[jibees]

When specifying version of jinja2 in requirements.txt, build is green ✅

[jibees]

AND IT'S GREEN ;)

Will clean the git history, and then think it's ready to be in code review ;)

[jibees]

Ok, build is broken. Probably a "false positive" (???), or a bad eyes for me.

[sigmundpetersen]

I don't think you ever got it green with nodejs > 14, did you?

[jibees]

Ok, with node_version at 15.x, we have this error:

 FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 102, in <module>\n  File \"<stdin>\", line 94, in _ansiballz_main\n  File \"<stdin>\", line 40, in invoke_module\n  File \"/usr/lib/python2.7/runpy.py\", line 188, in run_module\n    fname, loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 82, in _run_module_code\n    mod_name, mod_fname, mod_loader, pkg_name)\n  File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code\n    exec code in run_globals\n  File \"/tmp/ansible_npm_payload_bQEOIK/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 316, in <module>\n  File \"/tmp/ansible_npm_payload_bQEOIK/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 293, in main\n  File \"/tmp/ansible_npm_payload_bQEOIK/ansible_npm_payload.zip/ansible/modules/packaging/language/npm.py\", line 206, in list\n  File \"/usr/lib/python2.7/json/__init__.py\", line 339, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib/python2.7/json/decoder.py\", line 364, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib/python2.7/json/decoder.py\", line 382, in raw_decode\n    raise ValueError(\"No JSON object could be decoded\")\nValueError: No JSON object could be decoded\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Seems like upgrading node implies to upgrading also ansible?

[jibees]

I don't think you ever got it green with nodejs > 14, did you?

With node 14.x, I thought I had a green build, but let's try.

[sigmundpetersen]

I don't think you ever got it green with nodejs > 14, did you?

With node 14.x, I thought I had a green build, but let's try.

Yes, but not above 14 🙂

[jibees]

I don't think you ever got it green with nodejs > 14, did you?

With node 14.x, I thought I had a green build, but let's try.

Yes, but not above 14 🙂

Yes, that's exactly what I have in mind.

[dacook]

Ok, so in summary:

• Yarn latest version
• Node -- no change
• GH Actions latest version each
• Python latest version
• And lock to older version of Jinja2 (From memory, due to both Ansible 2.7 and our code)

I just double-checked, and the Ansible 2.7 docs confirm compatibility with Python 3.5+ ( https://docs.ansible.com/ansible/2.7/installation_guide/intro_installation.html#:~:text=Currently%20Ansible%20can%20be%20run,the%20BSDs%2C%20and%20so%20on. )

Question: Are there any dependencies that should also be updated in openfoodnetwork?
I don't think so (and maybe you already checked).

• Yarn seems to be implicitly latest version
• Node is already on a later version (hence this issue)
• GH Actions already latest version
• Python -- n/a
• Jinja2 -- n/a

Any documentation to update? Setting Python version is a manual setup task for ofn-install, but it already says 3.8.2. So this PR brings us in line with that (yay!).

So I think we're all good to go, except sadly we didn't get to do Node this time :(

Because we are changing yarn version, I think we should still do a dev test provision & deployment to vagrant (I will try to do this today), and also a staging provision & deployment for testing.

[dacook]

Tried testing dev deployment with vagrant today, but my VM has hung, so I will try again tomorrow after a system restart..

[jibees]

Ok, so in summary:

• Yarn latest version
• Node -- no change
• GH Actions latest version each
• Python latest version
• And lock to older version of Jinja2 (From memory, due to both Ansible 2.7 and our code)

Absolutely, thanks for sum-up, will update the description.

I just double-checked, and the Ansible 2.7 docs confirm compatibility with Python 3.5+ ( https://docs.ansible.com/ansible/2.7/installation_guide/intro_installation.html#:~:text=Currently%20Ansible%20can%20be%20run,the%20BSDs%2C%20and%20so%20on. )

Question: Are there any dependencies that should also be updated in openfoodnetwork? I don't think so (and maybe you already checked).

• Yarn seems to be implicitly latest version
• Node is already on a later version (hence this issue)
• GH Actions already latest version
• Python -- n/a
• Jinja2 -- n/a

I've wanted to update here because I've updated the ofn app:

openfoodfoundation/openfoodnetwork#9820

(realized that we could update to nodejs 18.x too...)

Any documentation to update? Setting Python version is a manual setup task for ofn-install, but it already says 3.8.2. So this PR brings us in line with that (yay!).

So I think we're all good to go, except sadly we didn't get to do Node this time :(

Yep. It's a half fail (but a half win too :p)

Because we are changing yarn version, I think we should still do a dev test provision & deployment to vagrant (I will try to do this today), and also a staging provision & deployment for testing.

Thanks for that!

[dacook]

Dev Test

1. New Vagrant VM: setup,provision,deploy ✅ (although I'm a little confused about seeding data and unable to login, but the app looks fine)
   local_vagrant : ok=189 changed=109 unreachable=0 failed=0 skipped=148 rescued=0 ignored=0
2. AU Staging: provision, deploy ✅
   staging.openfoodnetwork.org.au : ok=149 changed=4 unreachable=0 failed=0 skipped=129 rescued=0 ignored=0

I did a basic smoke screen test on staging: login and bulk edit a product. No errors in console, so I assume all the JS dependencies are installed fine.

@jibees do you think it necessary for further user testing? If not, good to go!

[jibees]

@jibees do you think it necessary for further user testing? If not, good to go

I think it's good then!
Let's merge ;) Thanks !