menuOmatic is an experimental AI command-line tool intended to help devs (and the dev) curious explore OpenAI's classification and Retrieval Augmented Generation (RAG) capabilities through the fun task of attempting to create menu plans.
It uses OpenAI's Chat API to classify pre-existing local recipe file information and a Retrieval Augmented Generation (RAG) process through the OpenAI Assistant API to utilise that information to generate flexible menu plans based on user preferences, dietary restrictions, etc.
AI based systems are hard to get right and the menuOmatic is an experimental app based on AI systems that can potentially provide recipe steps, ingredient lists and menus. As such the menuOmatic has a very serious potential to harm naive users if - and when - it goes wrong. For instance if the menuOmatic includes recipes that contain ingredients that a user is allergic to or hallucinates a processing step that results in undercooking and possibly food poisoning.
Therefore it is important for all menuOmatic users to understand that any menu plans, ingredient lists, recipes or any other output generated by this application is for informational and experimental purposes only.
And while the menuOmatic employs advanced AI to create plausible output, this is an experimental app and the user is solely responsible for reviewing and verifying the accuracy, suitability, and safety before relying on it.
In particular, the menuOmatic's output MUST never be relied upon to account for individual dietary restrictions, allergies, or medical conditions, and the generated content should not be considered a substitute for professional dietary, nutritional, or medical advice.
By using this application, you agree that the creators and developers of the application (including any and all third party IP or API providers) are not liable for any errors, omissions, or adverse outcomes resulting from the use of the application or its output.
It is the user's sole responsibility to exercise due diligence and make informed decisions when using any menuOmatic generated menu plan, recipe or any other output.
As an experimental app, the menuOmatic project welcomes constructive feedback, bug reports, and suggestions from the community. However, due to limited resources, support is currently focused on the macOS platform, with no guarantees of inter-version compatibility.
Contributions and collaborations from developers interested in expanding the project's functionality and platform compatibility are especially welcome.
The application supports a two stage classification and menu creation workflow process based on:
- Extracting classification information from a user's existing PDF recipe file collection.
- Feeding the user's classification information from 1. into a RAG process based on OpenAI Assistant's API to create menu plans based on it.
As well as a few system housekeeping utilites that have been found to be useful for cleaning up after experiments.
Bearing in mind the legal disclaimer, a rough indication of how well each of these capabilities currently seem to work with the shipped configuration based on informal testing with a 1,000 recipe testing corpus over the last couple of months is shown alongside each capability.
Objective: Extract recipes from Text based PDF files (image based PDF recipes files are not currently supported) and then performs automated extraction and classification of recipe information from them. Information extracted includes:
- Title - works well.
- Categorisation, e.g. Vegan, Vegetarian, Fish & Seafood, Poultry, Meat and Sweet - seems to work well.
- Meal Type, e.g. Snack, Main, Dessert - works well.
- Ingredient List - very limited manual testing, always manually cross check with the original recipe.
- Standardised Processing Steps - very limited manual testing, always manually cross check with the original recipe.
Objective: Create plausible menu plans considering various factors including:
- Ingredient delivery dates and freshness - work pretty well.
- Dietary preferences - works okay.
- Preparation times - works okay.
- Meal variety - over a weekly menu plan works okay.
- Previous menu selections - not sure, insufficient testing.
- Seasonality - insufficient testing.
- Special requests - works okay.
Overall for a weekly one main meal per day menu with once a week using a once per week delivery using a 1,000 available recipe corpus it subjectively creates menu's that form a reasonable starting point and save time versus the manual creation process.
Running the application creates a local SQLite DB on the user's machine and the following artefacts on Open AI's remote service:
- A File Object that contain a JSON serialised version of the recipe classification DB that is used to create ...
- A Vector Store, which is associated with ....
- An Assistant that communicates with the user through ...
- A message Thread
The application provides some assistance with the housekeeping and management of the Assistants and message Threads as the normal OpenAI web interface can be quite slow and cumbersome for managing these particularly in a developement situation.
NB: It is important to keep on top of the house keeping tasks as it can have a cost implication if ignored, see the section on OpenAI Costs.
- Unix like terminal environment, tested on on macOS 14 and 15, any modern Linux or similiar Unice should be fine but is untested.
- Python 3.13 or newer installation.
- OpenAI account with usage credits and an API access key (details on how to obtain and set this up can be found in many locations such as the one here)
- A good selection of Text based PDF recipe files proportional to the duration and complexity of the menu plans you intend to generate, i.e. unless you have a good selection of recipes, the AI will either generate repetitive and possibly nutritionally incomplete menu plans. PDF recipe files can be obtained from websites such as:
- Clone the repository:
cd some_directory_for_your_project_checkouts
git clone https://github.com/shufflingB/menuOmatic.git
cd menuOmatic- Create and activate a virtual environment NB: requires python 3.13 or newer.
python3.13 -m venv myenv
source myenv/bin/activate- Install the runtime dependencies into the virtual environment
pip install -r requirements.txt- If intending to develop, install the development dependencies
pip install -r requirements-dev.txtType annotation is used extensively and a working installation of is highly recommended for development. Details on how to manually install and use can be found here or it can be installed and integrated into VSCode with the Mypy Type Checker extension .
- Set up environment variables with the OpenAI API key:
export OPENAI_API_KEY=your_openai_api_keyThis can automated by adding a .env file to the root of the project with the following content:
OPENAI_API_KEY=your_openai_api_keyHelp is available for all sub commands by running ./menuOmatic.py --help or ./menuOmatic.py <sub-command> --help.
- Displays high level help information for the available commands.
- Allows the user to specify a custom database file location for holding recipe classifications and Assistant Thread information.
./menuOmatic.py --help
usage: menuOmatic.py [-h] [--dbFile DBFILE] {classify,planMenu,admin} ...
Recipe file extraction, classification and menu generation tool.
positional arguments:
{classify,planMenu,admin}
Available commands
classify Classify recipe files using AI and optionally export classifications
planMenu Generate a menu plan based on classified recipes
admin Manage menuOmatic's backend systems
options:
-h, --help show this help message and exit
--dbFile DBFILE SQLite database URL default: 'menuOmatic.db'The classify sub-command:
- Runs the AI classification on recipe files in the specified directory or dumps the existing classifications data in JSON format to the console.
- When classifying enables the following options:
- Forced reclassification of previously classified files.
- Specification a non-default directory to search for recipe files under
./menuOmatic.py classify --help
usage: menuOmatic.py classify [-h] [--forceReclassify [FORCERECLASSIFY ...]] [--exportOnly] [stemDir]
Use AI to extract information and classify recipe files recursively from a stem directory.
Results are cached in the database for future use. The classifier will:
- Extract recipe title and category
- List ingredients and steps
- Identify meal type (breakfast, lunch, dinner, etc.)
- Add any additional notes or warnings
Use --exportOnly to export existing classifications without performing new classifications.
positional arguments:
stemDir Directory to find recipe PDF files from. default: 'recipes'
options:
-h, --help show this help message and exit
--forceReclassify [FORCERECLASSIFY ...]
List of files to force reclassify
--exportOnly Export existing classifications as JSON without performing new classificationsUsage Example - classify all recipes under a non-default stem directory and store classification information in non-default db file
./menuOmatic.py --dbFile some_test.db classify /path/to/pdf_recipe_files_stem_directorySequentially compares the paths of files under the stem directory /path/to/pdf_recipe_files_stem_directory with those recorded in the DB some_test_.db. Those with no matching record will have their text extracted and be classified via the OpenAI chat interface.
Notes
- To force reclassification of a recipe give the full path to file to the
--forceReclassifyoption or delete the record from thesome_test.dbDB file using sqlite tooling. - To remove a recipe, delete the recipe file and then delete the record from the
some_test.dbDB file using sqlite tooling.
The planMenu sub command creates a menu plan based on the classified recipes by uploading the classification data to an OpenAI Assistant and using that to generate a menu plan.
It allows the user to:
- Specify a non-default OpenAI Assistant to use for the menu plan generation.
- Send special requests to the AI Assistant, e.g. to specify a non-default menu plan duration, different food delivery dates, dietary preferences, etc.
- Acknowledge that the previous menu suggestion was used, so the Assistant should update its records accordingly.
./menuOmatic.py planMenu -h
usage: menuOmatic.py planMenu [-h] [--specialRequest SPECIALREQUEST] [--accepted] [--assistantName ASSISTANTNAME]
Create a weekly menu plan using the classified recipes in the database.
The planner considers:
- Dietary preferences and restrictions
- Meal variety and balance
- Preparation time constraints
- Previous menu selections
- Special requests (if provided)
Use --specialRequest to add specific requirements for this plan.
Use --accepted to confirm a menu plan was used (helps with future planning).
options:
-h, --help show this help message and exit
--specialRequest SPECIALREQUEST
Arbitrary string specifying special requests to pass to the Assistant
--accepted Indicate that the user has accepted the previous menu suggestion, so the Assistant should update its records accordingly
--assistantName ASSISTANTNAME
Name of the Open AI Assistant to use, default: 'menuOmatic_planning_assistant' ./menuOmatic --dbFile some_test.db planMenu --assistantName some_assistant_name --specialRequest "Please can we have a vegan recipe on Friday" > ./some_menu_plan.json - Iff necessary:
- Uploads changed classification database 'some_test.db' as an OpenAI File Object and uses it to create an OpenAI Vector Store object.
- Creates or updates an OpenAI Assistant named
some_assistant_namewith the Vector Store object.
- Uses the Thread messaging mechanism to request that the Assistant
some_assistant_nameuses the Vector Store object to create a menu plan that takes into the special menu request option"Please can we have a vegan recipe on Friday" - Dumps the resultant suggested menu plas as JSON to stdout that is redirected into a file
./some_menu_plan.json.
Use ./menuOmatic.py admin -h, ./menuOmatic.py admin threads -h and ./menuOmatic.py admin assistants -h
./menuOmatic.py admin --assistantName some_assistant_name assistants updateUpdate the assistant named some_assistant_name to pick up any changes made in the menu planning prompt/system instructions.
Using OpenAI's API's is not free and normal operation involves the application using Chat to classify recipe texts via the Chat API and the Assistants API to perform Menu generation.
As of January 2025, understanding if the system can be made to work at all is the main motivation for the application and so by default the app is configured in app_constants.py to use the most capable/expensive model ('gpt-4o') for both the Chat and Assistant API's
The system uses the Chat API to upload the extracted text from the recipe PDF's and extract data from them, this results in costs that are approximately of the order of Dollar's per 1,000 Recipe PDFs
These costs can mount up quickly if performing a large number of classifications, therefore when experimenting with classification it is both quicker and more economical to devise and use small recipe subsets to prove out the work.
Menu generation uses the Assistant's API which involves the creation of a permanent OpenAI hosted:
- Vector Store from the extracted Classification Databases
- An Assistant that uses the Vector Store to supplment its basic model information.
- And a Thread that is used to store the record of the system's conversation with the model.
Creation, storage and inference of all of these is much less expensive than Classification, typically Cents per user interaction
However, these Assistants, their Vector Store, File Object and conversational Threads are stored on OpenAI's Cloud infrastructure by default in perpetuity and this will have a small reoccuring cost unless they are manually deleted.
Therefore is important to monitor to ensure that no surplus artefacts are being inadvertanly created, particularly when running the Tests or other experiements.
These can monitored for:
- Assistants https://platform.openai.com/assistants/
- Vector Store https://platform.openai.com/storage/vector_stores
- Files Object associated with the Vector Stores https://platform.openai.com/storage/files
- Thread conversations with the Assistants https://platform.openai.com/threads
Most of these objects use a combination of semantic naming and meta-data to aid understanding and make administration somewhat easier.
Application defaults such as the OpenAI Assistant name and the database file location are stored in the .../src/app_constants.py file.
How the system requests the AI to classify recipes is controlled by the classification prompt RECIPE_CLASSIFIER_PROMPT which is defined in the .../src/recipe_classification.py file.
Change here can be used to change the classification of:
- Meal Type e.g. Breakfast, Lunch, Dinner, etc
- Category e.g. Vegan, Vegetarian, Poulry, Meat-based, Sweet, Fish and Seafood, etc.
How the system requests the AI generate a menu plan is controlled by the menu planning prompt. This is sent as a system instruction to the Assistant either on creation or on update. The current value is stored in the file .../src/assistant_instructions.py
User preferences are stored in the .../src/user_preferences.py file. The details in this file are sent as the Thread instantiation Message to the Assistant on first run or admin forced update.
Unfortunately it's not possible to legally distribute the corpus of downloaded PDF recipe files that has been used to test the application development. Therefore it would be nice to either put together a corpus of PDF files that could legally be used for testing and as a starter pack for users or provide a Selenium script that downloaded for the user.
At the moment each PDF file is classified in serial which is very slow when performing initial classification run on recipe file corpus with more than about 20 recipes. For example with 1,000 recipes it was taking between 1/2 to 1 hour during December 2024.
NB: After the first run, it will only attempt to classify and add (new) recipes for which it does not have a filename record in the DB.
At the moment, only way to remove recipe classification that are no longer required is to use SQLite tooling and manually delete the record from the DB and then run planMenu to update the VectorStore with the updated DB data
At the moment, the menu plan is essentially a dump of the JSON response from the Assistant. This really needs to be tidied up and information that is effectively duplicated (such as path - it's already in the classification DB) - removed.
Make finding the recipes in the menu plan easier for those following the plan.
Needs automated testing to ensure reliable performance, that in turn requires a corpus of ingredient labelled PDF recipes.
Use OpenAI to extract recipe information from photo's of recipes embedded in PDF files.
At the moment the prompts (classification, generation and user preferences) and are just part of the application's Python codebase. Ideally each of these should be in their own independently validated domain specific configuration file.
Would be nice to enable the use of alternative - perhaps local - non-OpenAI AI models for both the classification and menu based generation tasks.