Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add response headers and nginx logging to identify requests and registered applications used for a request #3368

Closed
sarayourfriend opened this issue Nov 19, 2023 · 0 comments · Fixed by #3369
Assignees
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: api Related to the Django API

Comments

@sarayourfriend
Copy link
Collaborator

Problem

Right not it's not possible for us to identify applications related to a request. If any particular registered application is causing issues, it is essentially impossible for us to identify that through our existing request logging.

Description

Add the following parameters to Nginx logging (some will require new response headers added to relevant requests):

  • Request ID (from the request ID logging middleware, present in the x-request-id response header)
  • Registered client application name and verified status

The last two will require new headers added so that we can log them in Nginx.

This is not a privacy concern. Our API terms of service already allow for this kind of introspection into registered application usage, and we theoretically have more granular data if we really wanted to fingerprint or identify individual users. To clarify, that is not our intention with this. If we wanted to do that we'd fingerprint off of user agent and source IP. The intention behind this is to help identify potentially nefarious API requesters and to see if any of the identifiable bot traffic we see in traffic patterns is tied to specific registered applications. I'm adding this caveat just to clarify that adding this logging does not make anything less private than it already is.

@sarayourfriend sarayourfriend added 🟨 priority: medium Not blocking but should be addressed soon 🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: api Related to the Django API labels Nov 19, 2023
@sarayourfriend sarayourfriend self-assigned this Nov 19, 2023
@github-project-automation github-project-automation bot moved this to 📋 Backlog in Openverse Backlog Nov 19, 2023
@openverse-bot openverse-bot moved this from 📋 Backlog to ✅ Done in Openverse Backlog Nov 26, 2023
@openverse-bot openverse-bot moved this from ✅ Done to 📋 Backlog in Openverse Backlog Nov 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: api Related to the Django API
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant