Purpose

Creates image descriptions, captions, titles, and alt tags for image attachments in WordPress using a local large language model.

No data is sent to external services. All data is processed on the local machine.

In a WordPress context, types of text associated with images have these uses:

• Alt: Displays on images as an image is loading. Relevant for accessibility, for example, screen readers, and search engines.
• Title: Displays as a tooltip on mouse hover. May also be read by screen readers depending on user settings.
• Description: Displays on image attachment page in WordPress themes. Also useful for search.
• Caption: Displays below the image in the page layout.

Requirements

This plugin assumes WordPress is running on a local installation, Ollama.ai is installed, and you have access to wp-cli.

Ollama currently supports macOS and Linux, with a Windows version in progress. Testing was conducted on a 2012 Intel Mac mini and a 2022 M1 Mac mini. Intel processors will process images using the CPU, while Apple Silicon chips will make use of the Neural Engine, running much faster without using the main processor resources.

Install Ollama

Graphically: Download the app from Ollama.ai. Run the app, and click the button at launch to install command line tools.

With Homebrew:

brew install ollama

Use Ollama

Download a vision model

ollama pull llava

Run the Language Model server

Either open the Ollama app, which will appear in the menu bar, or run in Terminal:

ollama serve

Install wp-cli

wp-cli comes bundled with Local WP. It can also be installed with Homebrew:

brew install wp-cli

Install this plugin

Option 1: Add this plugin folder to wp-content/plugins and activate it.

Option 2: Place the main PHP file in wp-content/mu-plugins/describe-media-library.php

Generate Image Descriptions

Once Ollama is running, image alt tags, descriptions, captions, and titles can be generated by running the plugin's command within a directory located in the WordPress installation:

wp _medic describe-images

...this will output descriptions to the terminal and save a CSV to wp-content/uploads/image-descriptions.csv.

Once there are descriptions in the CSV, the file may be reviewed or revised, then written to the WordPress database with:

wp _medic describe-images --write-to-db

Uploading the plugin and CSV to a live site and running the above command provides a way to update images descriptions on a production installation without migrating a database.

Modify Description Guidelines

When generating, these are the default inputs:

wp _medic describe-images \
   --alt="Please write a thorough description of the image in a format appropriate for an alt tag focused on accessibility." \
   --description="Please write a comma-separated list of keywords and relevant synonyms related to this image, focusing on relevancy for search." \
   --caption="Please write a descriptive caption for this image appropriate for displaying to a user reading an article where the image is referenced." \
   --title="Please write a short title for this image."

These can be overriden to anything else:

wp _medic describe-images \
	--alt="Describe the image for an accessible alt tag in French." \
	--description="Write a comma-separated list of keywords and topics related to this image." \
	--caption="Write a caption for this image in the form of a Haiku." \
	--title="Describe this image using only emoji."

Or leave the argument blank to skip it.

e.g., to write only alt tags:

wp _medic describe-images \
	--alt="Please write a thorough description of the image in a format appropriate for an alt tag focused on accessibility." \
	--description \
	--caption \
	--title

Image Size

The default image size passed to the model is large. Any defined WordPress image size may be used, such as thumbnail, medium, large, or full. In testing on a Quad-Core Intel i7 Mac mini using CPU, large said it would take around 24 days to generate all 4 description types for 1,600 images. However, switching to medium produced descriptions which seemed fabricated, so the default is large.

This will go slower, but is the default due to accuracy:

wp _medic describe-images --image_size="large"

This will go faster, but accuracy may degrade:

wp _medic describe-images --image_size="medium"

If doing this at scale, one would ideally have access to hardware with Apple Silicon, NVIDIA GPUs, or Cloud TPUs. Here, we make use of older hardware by designing systems which distribute tasks across many machines.

Collaborate with 🌦️️🩺️ Cloud Medic

Do you have a project involving artificial intelligence, data engineering, or medicine you would like to bring to life?

Connect with us at make@cloudmedic.io

Please use subject Collaboration and share short description of how we can help bring your ideas to reality.