Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude media from sources without ContentSource record from search #4076

Open
obulat opened this issue Apr 9, 2024 · 1 comment · Fixed by #4238
Open

Exclude media from sources without ContentSource record from search #4076

obulat opened this issue Apr 9, 2024 · 1 comment · Fixed by #4238
Assignees
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: api Related to the Django API

Comments

@obulat
Copy link
Contributor

obulat commented Apr 9, 2024

Problem

When we add a new provider, we don't immediately add a ContentSource record to the API. This means that the media is immediately available in search results.

Description

When building an excluded source query, we should add the sources that don't have a ContentSource model to the excluded sources. To do that, we can get all sources using:

sources = cache.get(key=source_cache_name)

This is where the query is built:

if not filtered_sources:
filtered_sources = list(
models.ContentSource.objects.filter(filter_content=True).values_list(
"source_identifier", flat=True
)
)

Alternatives

An alternative would be to add a step to the data refresh that would add new ContentSource records for all the sources that were added during the data refresh: #3548

Additional context

#2487 is related in that it added a filter for the single results: if "Hide content" for their provider is set to True, the result is not returned.

@obulat obulat added 🟨 priority: medium Not blocking but should be addressed soon 🛠 goal: fix Bug fix 💻 aspect: code Concerns the software code in the repository 🧱 stack: api Related to the Django API labels Apr 9, 2024
@openverse-bot openverse-bot moved this to 📋 Backlog in Openverse Backlog Apr 9, 2024
@openverse-bot openverse-bot moved this from 📋 Backlog to 🏗 In Progress in Openverse Backlog Apr 30, 2024
@krysal krysal self-assigned this May 7, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to ✅ Done in Openverse Backlog May 21, 2024
@krysal
Copy link
Member

krysal commented May 31, 2024

Reopening since the #4238 was reverted.

@krysal krysal reopened this May 31, 2024
@openverse-bot openverse-bot moved this from ✅ Done to 📋 Backlog in Openverse Backlog May 31, 2024
@obulat obulat changed the title Exclude media from sources without ContentProvider record from search Exclude media from sources without ContentSource record from search Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: api Related to the Django API
Projects
Status: 📋 Backlog
Development

Successfully merging a pull request may close this issue.

2 participants