Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add database cleanup functions for transactions and vertex builds #4694

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

ogabrielluiz
Copy link
Contributor

This pull request introduces functions to clean up old transactions and vertex builds in the database, ensuring that only a configured maximum number of each is retained. The clean_transactions function deletes transactions exceeding the specified limit, while clean_vertex_builds performs a similar cleanup for vertex builds. Additionally, these cleanup functions are integrated into the service initialization process, and new configuration options for maximum retention limits have been added.

…atabase

- Implement `clean_transactions` to delete transactions exceeding the configured limit.
- Implement `clean_vertex_builds` to delete vertex builds exceeding the configured limit.
- Integrate cleanup functions into the service initialization process.
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels Nov 19, 2024
@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Nov 19, 2024
- Wrap transaction and vertex build cleanup operations in try-except blocks.
- Log success and error messages for cleanup operations.
- Rollback session on exceptions without re-raising, as these are cleanup tasks.
- Adjust service initialization order to ensure proper setup.
Copy link

codspeed-hq bot commented Nov 19, 2024

CodSpeed Performance Report

Merging #4694 will degrade performances by 36.25%

Comparing transaction-vb-limit (52aeb3d) with main (2fa2580)

Summary

⚡ 4 improvements
❌ 2 regressions
✅ 9 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main transaction-vb-limit Change
test_create_starter_projects 965.9 ms 698.9 ms +38.2%
test_initialize_services 424.2 ms 179.1 ms ×2.4
test_initialize_super_user 670.1 ms 425.8 ms +57.4%
test_load_flows 2.2 ms 1.3 ms +75.02%
test_successful_run_with_input_type_text 140.7 ms 217 ms -35.16%
test_successful_run_with_output_type_debug 139.1 ms 218.2 ms -36.25%

async def initialize_services(*, fix_migration: bool = False) -> None:
"""Initialize all the services needed."""
# Test cache connection
get_service(ServiceType.CACHE_SERVICE, default=CacheServiceFactory())
# Setup the superuser
await asyncio.to_thread(initialize_database, fix_migration=fix_migration)
async with get_db_service().with_async_session() as session:
await setup_superuser(get_service(ServiceType.SETTINGS_SERVICE), session)
settings_service = get_service(ServiceType.SETTINGS_SERVICE)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alignment of the try block seems incorrect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants