An implementation of the model proposed by Translating Math Formula Images to LaTeX Sequences Using Deep Neural Networks with Sequence-level Training based off of this implementation of an older model by @luopeixiang. The model takes in images of mathematical formulas as input and outputs their corresponding LaTeX markup using a CNN encoder and LSTM decoder.
| BLEU-4 | Edit Distance | Exact Match |
|---|---|---|
| 0.8932 | 0.9226 | 0.3401 |
The dataset containing images and their corresponding ground-truth sequences was sourced from here. Preprocessing was done through the process described in this repository by @untrix.
Only follow steps 2-3 if you want to train a custom model instead of using the one in the latest release.
pip install -r model/requirements.txt
If you want to use my trained model along with the api provided in app.py, download the assets in the latest release and store them in a subdirectory of /models called utils.
Download the dataset from the link in the Data and Preprocessing section of this document.
Run the following:
python train.py \
data_dir=[path to directory containing df_train, df_valid files] \
image_dir=[path to directory containing all images] \
output_dir=[path to directory where checkpoints should be saved] \
vocab_path=[path to vocab mapping from integer ids to tokens as a .pkl file] \
--cuda=[include only if training on gpu]