[7806] plans and projects api caching

[m4ra]

Tasks

• PR name contains story or task reference
• Documentation (docs and inline)
• Tests (including n+1 and django_assert_num_queries where applicable)
• Changelog

[goapunk]

Some initial comments and the question about how we handle cache expiry times

[m4ra]

@goapunk have another look when you can. I need to add a test, e.g adding future projects and check if the are added in the cache.

[goapunk]

Nice, looks good to me so far, I left some comments with questions. I'm not sure about the failing tests, seems to be a bit tricky with the combination of redis + CELERY_TASK_ALWAYS_EAGER=True. Without it we need a running celery worker though

[m4ra]

@goapunk there was a typo in the reset cache task of the key to be deleted. As I was testing locally and because of the asterisks I was using, something was not captured. Anyhow I simplified the tests and removed the API calls as these are part of the test_api_caching. Also I ended up using a 10min check and a list of projects that will become past same as for future projects, otherwise we may miss any other expiring between the 10mins if one is already set for expiration. Looking again at the tasks and functions I wonder if they should be consolidate instead of having separated for future and past projects.

[goapunk]

@goapunk there was a typo in the reset cache task of the key to be deleted. As I was testing locally and because of the asterisks I was using, something was not captured. Anyhow I simplified the tests and removed the API calls as these are part of the test_api_caching. Also I ended up using a 10min check and a list of projects that will become past same as for future projects, otherwise we may miss any other expiring between the 10mins if one is already set for expiration. Looking again at the tasks and functions I wonder if they should be consolidate instead of having separated for future and past projects.

I guess the only difference is the deletion of projects_futureParticipation vs projects_pastParticipation, so yeah, maybe we just delete them both and consolidate the code? What do you think?

[m4ra]

@goapunk I merged the two scheduled tasks, the actual resets for future/past projects i left as it is, since it will save us from queering either future or past projects again via the url, which are time/power consuming.
I added also django query checks in the tests, very helpful and eye opening, as without cache, the api endpoint queries 12 times (with the prefetch-related).

[goapunk]

Really nice! Just some small comments left

[goapunk]

@goapunk I merged the two scheduled tasks, the actual resets for future/past projects i left as it is, since it will save us from queering either future or past projects again via the url, which are time/power consuming. I added also django query checks in the tests, very helpful and eye opening, as without cache, the api endpoint queries 12 times (with the prefetch-related).

@m4ra yeah, the query check is really helpful, we definitely need to work on improving the queries/structure. Ideally we will have checks in all our tests in the future 🤞

[m4ra]

@goapunk done!

[goapunk]

Really cool :) Before we deploy it we need to make some changes in the admin repo