MoviesAPI

Requirements

1. Fetch test data via https from OMDB API

• Fetch 100 movies from the OMDB API
• Movies should be saved in the database.
• Run this method once, only if the database is empty.

2. API Implementation

• The api should have a method that returns a list of movies from the database
• There should be option to set how many records are returned in single API Response (by default 10)
• There should be pagination implemented in the backend
• Data should be ordered by Title
• The api should have a method that returns a single movie from the database
• There should be option to get the movie by title
• The api should have a method to add a movie to the database
• Title should be provided in request
• All movie details should be fetched from OMDB API and saved in the database
• The api should have a method to remove a movie from the database
• There should be option to remove movie with it's id
• This method should be protected so only authorized user can perform this action

3. Unit tests for all cases (mock everything else)

4. GCP deployment

PROJECT STRUCTURE

.
├── alembic.ini                   # Configuration file for Alembic
├── common                        # Directory containing common utility functions and benchmarking scripts
│   ├── benchmark.py              # Module for benchmarking performance
│   ├── performance_fixtures.py   # Module containing fixtures for performance testing
│   ├── tests                     # Directory containing test scripts for utility functions
│   └── utils.py                  # Common utility functions
├── data_fetcher                  # Directory containing modules related to fetching movie data
│   ├── fetcher_config.json       # Configuration file for the movie data fetcher
│   ├── movie_data_fetcher.py     # Module for fetching movie data from external sources
│   └── tests                     # Directory containing test scripts for the movie data fetcher
├── data_layer                    # Directory containing modules related to the data layer
│   ├── models.py                 # Module defining SQLAlchemy models for database tables
│   ├── movies_repository.py      # Module containing the repository class for accessing movie data from the database
│   ├── tests                     # Directory containing test scripts for the data layer modules
│   └── unit_of_work.py           # Module defining the unit of work pattern for managing database transactions
├── Dockerfile                    # Configuration file for building a Docker image of the application
├── docs                          # Directory for storing documentation files (Generated from docstrings in the code)
├── htmlcov                       # Directory containing coverage reports generated by test coverage tools
├── migrations                    # Directory containing database migration scripts generated by Alembic
├── movies.db                     # SQLite database file containing movie data (Initialized at the start of the app)
├── movies_service                # Directory containing modules and tests for the main application logic
│   ├── api                       # Directory containing API-related modules
│   │   └── api.py                # Module defining the API endpoints using FastAPI
│   ├── app.py                    # Module containing the main FastAPI application instance
│   ├── oas.yaml                  # OpenAPI Specification file defining the API endpoints
│   └── tests                     # Directory containing test scripts for the application modules
├── README.md                     # Markdown file containing information about the project
├── requirements.txt              # File listing Python dependencies required for the project
└── dev-requirements.txt          # File listing additional Python dependencies required for development and testing purposes

Project Architecture Overview

Unit of Work and Repository Pattern

The project employs the Unit of Work and Repository patterns to promote a clean and modular architecture for managing data interactions within the application. These patterns offer several benefits:

Decoupling of Data Access Logic:

• The Unit of Work pattern encapsulates the logic for managing transactions and database sessions, allowing for centralized control over data persistence operations.
• The Repository pattern abstracts away the details of database interactions, providing a uniform interface for accessing and manipulating data entities.

Together, they promote separation of concerns and facilitate easier maintenance and testing of data access code.

Improved Testability: By decoupling data access logic from business logic, the application becomes more testable. Unit tests can be written for repository classes with mock implementations, ensuring that data access functionality behaves as expected without the need for database connectivity.

Enhanced Code Reusability: Repositories provide a consistent API for interacting with data entities, making it easier to reuse data access logic across different parts of the application. This reduces code duplication and promotes a more modular and maintainable codebase.

Models as Data Layer

The project utilizes models to represent the structure and behavior of data entities within the application.

The Models defines Python classes that represent the data entities (movies). Each class corresponds to a table in the database, and attributes of the class represent columns in that table.

Models are commonly used in conjunction with Object-Relational Mapping (ORM) libraries like SQLAlchemy

Models offer several advantages:

Data Integrity and Validation: Models enforce business rules and constraints, ensuring that data stored in the database meets certain criteria.

Abstraction of Database Interactions: Models abstract away the details of database interactions, allowing developers to work with domain-specific objects rather than directly dealing with database queries and operations. This separation of concerns simplifies the codebase and makes it more maintainable.

Organization and Structure: Models organize and structure the data entities within the application, defining relationships between different entities and providing a clear representation of the data schema. This organization facilitates efficient querying and retrieval of data.

Data Fetching in a Separate Module

The project adopts a modular approach by separating data fetching logic into a separate module. This separation offers the following benefits:

Code Organization and Maintainability: By isolating data fetching functionality into its own module, the codebase becomes more organized and easier to maintain. Developers can focus on specific areas of functionality without being overwhelmed by the complexity of data fetching logic.

Reusability and Scalability: Separating data fetching logic into reusable modules promotes code reusability and scalability. The module can be easily reused across different parts of the application or extended to support additional data sources or APIs, making it more adaptable to changing requirements.

---

Project Features

Test Coverage

• The project codebase aims to achieve 100% test coverage to ensure comprehensive testing of all functionality and minimize the risk of bugs and regressions.

Documentation

• Documentation for the project is automatically generated using pdoc3 and can be found in the docs/ folder in HTML format. This documentation provides detailed information about the project's modules, classes, functions, and their usage.

Performance Monitoring

• The project includes performance fixtures to measure CPU and memory usage as well as execution time for various operations. This helps identify potential performance bottlenecks and optimize resource utilization.

Improved Data Fetching Performance

• Initial Version: The first version of the project took approximately 270 seconds to fetch data for 100 movies from external sources during database initialization.

• Optimized Version: After implementing parallel request optimization, the data fetching time for the same 100 movies was reduced to approximately 25 seconds. This significant improvement in performance enhances the overall efficiency of the application and improves the user experience.

Benchmarking

• Benchmarking script is implemented to measure the API response performance and evaluate the effects of the implemented cache system. This allows for continuous performance monitoring and optimization to ensure optimal API responsiveness.

• Cache Off:
  Average response time for /movies endpoint : ~0.014 seconds.
  Average response time for /movie/The Wonderful World of Disney: 40 Years of Television Magic : ~0.013 seconds.

• Cache On: Average response time for /movies endpoint : ~0.0024 seconds.
  Average response time for /movie/The Wonderful World of Disney: 40 Years of Television Magic : ~0.0015 seconds.

/movies response is ~5.8x faster with caching. /movies/{movie_title}: response is ~8.7x faster with caching.

Authorization for Movie Deletion

• Authorization using an API key is required for deleting movies. This security measure ensures that only authorized users can perform sensitive operations, enhancing the security of the application.

SIMPLIFICATIONS

• External OMDb Api Key kept in data_fetcher config json file.
• MoviesAPI Api Key validation hardcoded. Valid api keys should be kept encoded in a database.