example-code.ipynb

{
 "nbformat": 4,
 "nbformat_minor": 0,
 "metadata": {
  "colab": {
   "name": "ExtractTable Usage -2.2.0",
   "provenance": [],
   "collapsed_sections": [],
   "toc_visible": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "<a href=\"https://colab.research.google.com/github/ExtractTable/ExtractTable-py/blob/master/example-code.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 1. Installation"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "!pip install -U ExtractTable"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 2. Import and check version"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "from ExtractTable import ExtractTable"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "print(ExtractTable.VERSION)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 3. Create Session & Validate API Key\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.1 **Create Session** with your API Key"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "api_key = YOUR_APIKEY_HERE"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "et_sess = ExtractTable(api_key)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# et_sess.__dict__"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.2 **Validate** the Key and check the plan usage"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "usage = et_sess.check_usage()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "*If there is no error encountered in the above cell, it means we have a valid API key. Now, lets get started by checking the usage and trigger the file for processing*"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# et_sess.server_response"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "print(usage)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.3 Check Usage Details\n",
    "\n",
    "**credits**: Total number credits attached to the API Key\n",
    "\n",
    "**queued** : Number of triggered jobs that are still processing in the queue\n",
    "\n",
    "**used**   : Number of credits already used "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 4. Trigger the extraction process\n",
    "\n",
    "> Note: We will use the session, `et_sess`, created earlier in step 3.1, to save the session data and retrieve when needed"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4.1 Accepted Input Types\n",
    "\n",
    "**Allowed input formats** are:\n",
    "- Image\n",
    "  - JPG/JPEG\n",
    "  - PNG\n",
    "- PDF\n",
    "  - Text PDF\n",
    "  - Scan PDF\n",
    "  - Image PDF\n",
    "\n",
    "\n",
    "**Input Location Options**\n",
    "- Location can be a file from the local drive\n",
    "- Accessible remote URL - *the file object will be locally downloaded and deleted once sent to the process*"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# image_location = \"local_image_path_OR_remote_image_url_with_tables\"\n",
    "# image_location = r'samples/BlurryImage.jpg'\n",
    "image_location = \"https://raw.githubusercontent.com/ExtractTable/ExtractTable-py/master/samples/QualityImage.jpg\""
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4.2 Process an <ins>IMAGE</ins> Input\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "table_data = et_sess.process_file(filepath=image_location, output_format=\"df\")"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "table_data"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4.3 Process a <ins>PDF</ins> Input"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# pdf_location = \"local_image_path_OR_remote_image_url_with_tables\"\n",
    "# pdf_location = r'samples/BlurryImage.jpg'\n",
    "pdf_location = \"https://raw.githubusercontent.com/ExtractTable/ExtractTable-py/master/samples/QualityImage.jpg\""
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "table_data = et_sess.process_file(filepath=Location_of_PDF_with_Tables, pages=\"all\", output_format=\"df\")"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Below are the sample values ```pages``` accepts **string** type\n",
    "\n",
    "\n",
    "\n",
    "| pages  \t| Explanation                                     \t|\n",
    "|----------\t|-------------------------------------------------\t|\n",
    "| \"1\"      \t| [Default] considers only 1st page of the PDF    \t|\n",
    "| \"1,3,5\"  \t| considers only 1st, 3rd and 5th page of the PDF \t|\n",
    "| \"1, 3-5\" \t| considers 1st, 3rd, 4th and 5th page of the PDF \t|\n",
    "| \"all\"    \t| considers complete PDF                          \t|"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4.4 Table Output options\n",
    "\n",
    "> By default, the `process_file()` returns **only** the table data. Output depends on the `output_format` , explained below\n",
    "\n",
    "Explore the available options with `ExtractTable._OUTPUT_FORMATS`\n",
    "\n",
    "| output_format \t| Explanation                                \t|\n",
    "|---------------\t|--------------------------------------------\t|\n",
    "| \"df\"          \t| [Default] Array of Pandas dataframes        \t|\n",
    "| \"dataframe\"   \t| same as \"df\"; Array of Pandas dataframes    \t|\n",
    "| \"json\"        \t| JSON data with index orientation           \t|\n",
    "| \"dict\"        \t| Similar to JSON data but python dictionary \t|\n",
    "| \"csv\"         \t| Array of locally saved CSV file locations   \t|\n",
    "| \"xlsx\"         \t| To save multiple tables as sheets into a single excel\t|\n",
    "| \"excel\"         | same as \"xlsx\"; output is an array of excel location\t|\n",
    "\n",
    "\n",
    "Default output is an array of pandas dataframes, with which you can change to any other format like excel, html etc. Follow https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4.5 Explore session objects\n",
    "\n",
    "> **Explore** all objects of the latest file processing with `et_sess.__dict__.\n",
    "keys()` - Depends on the plan type of your API Key"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "et_sess.__dict__.keys()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Based on the API Key PLAN type, the et_sess contains below objects\n",
    "\n",
    "| Object          \t| Explanation                                                                            \t|\n",
    "|-----------------\t|----------------------------------------------------------------------------------------\t|\n",
    "| api_key         \t| Your API Key                                                                           \t|\n",
    "| _session        \t| Session data of the **latest** performed request/action                                          \t|\n",
    "| input_filename  \t| Name of the processed input file |\n",
    "| ServerResponse  \t| Complete ServerResponse, along with response code and headers                          \t|\n",
    "| server_response \t| complete server response content; equivalent to `ServerResponse.json()`                \t|\n",
    "| JobStatus       \t| Job Status of the triggered process                                                    \t|\n",
    "| Pages           \t| Number of pages in the input; also number of credits consumed on the triggered process \t|\n",
    "| Tables          \t| Tabular Data in JSON format with index orientation; ordered table wise                 \t|\n",
    "| Lines           \t| Text Data in JSON format, ordered page wise                                            \t|"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4.6 Save Table & Text to LOCAL\n",
    "\n",
    "```python\n",
    "et_sess.save_output(output_folder, output_format=\"csv\")\n",
    "```\n",
    "`output_format` param is relavant only for the table data, with options \"csv\" or \"xlsx\"\n",
    "\n",
    "\n",
    "> Note: As the `et_sess` contains the latest action performed, make sure this call is right after the `process_file()`"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 5. Explore the Output"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 5.1 Output Structure\n",
    "\n",
    "> **Understand the output**: The response of a triggered job is a JSON object in the below format. \n",
    "\n",
    "Note that the response depends on the plan type of the API Key.\n",
    "\n",
    "\n",
    "```javascript\n",
    "{\n",
    "    \"JobStatus\": <string>,                              # Status of the triggered Process  @ JOB-LEVEL\n",
    "    \"Pages\": <integer>,                                 # Number of pages processed in this request @ PAGE-LEVEL\n",
    "    \"Tables\": [<list of key-value objects of table>     # List of all tables found @ TABLE-LEVEL\n",
    "        {\n",
    "            \"Page\": <integer>,                              ## Page number in which this table is found\n",
    "            \"CharacterConfidence\": <float>,                 ## Accuracy of Characters recognized from the input-page\n",
    "            \"LayoutConfidence\": <float>,                    ## Accuracy of table layout's design decision\n",
    "            \"TableJson\": <dict>,                            ## Table Cell Text in key-value format with index orientation - {row#: {col#: <str>}}\n",
    "            \"TableCoordinates\": <dict>,                     ## Top-left & Bottom-right Cell Coordinates - {row#: {col#: <list(x1,y1,x2,y2)>}}\n",
    "            \"TableConfidence\": <dict>                       ## Cell level accuracy of detected characters - {row#: {col#: <float>}}\n",
    "        },\n",
    "    {...}                                               ## ... more \"Tables\" objects\n",
    "    ],\n",
    "    \"Lines\": [<list of key-value objects>               # Pagewise Line details @ PAGE-LEVEL\n",
    "        {\n",
    "            \"Page\": <integer>,                          # Page number in which the lines are found\n",
    "            \"CharacterConfidence\": <float>,             # Average Accuracy of all Characters recognized from the input-page\n",
    "            \"LinesArray\": [\n",
    "                <list of key-value objects of line>     # Ordered list of lines in this page @ LINE-LEVEL\n",
    "                {\n",
    "                    \"Line\": <str>,                          ## Detected text of the complete line\n",
    "                    \"WordsArray\": [\n",
    "                        <list of key-value objects>         ## Word level datails in this line @ WORD-LEVEL\n",
    "                        {\n",
    "                            \"Conf\": <float>,                    ### Accuracy of recognized characters of the word\n",
    "                            \"Word\": <str>,                      ### Detected text of the word\n",
    "                            \"Loc\": [x1, y1, x2, y2]             ### Top-left & Bottom-right coordinates, w.r.t the input-page width-height dimensions\n",
    "                        },\n",
    "                    {...}                                   ### More \"WordsArray\" objects\n",
    "                    ]\n",
    "                },\n",
    "            {...}                                       ## More \"LinesArray\" objects\n",
    "            ]\n",
    "        },\n",
    "    {...}                                               # More Pagewise \"Lines\" details\n",
    "    ]\n",
    "}\n",
    "```"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 5.2 Output Details\n",
    "\n",
    "Output objects are based on the API Key Plan type. Available plan types are \n",
    "\n",
    "**Purchased Plans**\n",
    "*   \"LITE\"   - **only table data** in the output\n",
    "*   \"FULL\"   - **table and text data** in the output\n",
    "*   \"EXTRA\"  - **table, text data along with cell & word coordintates and character detection accuracy**\n",
    "\n",
    "**Promotional Plans**: Any plan other than Purchased plans are promotional\n",
    "*   \"free_trial\", \"camelotpro\" - these are promotional API Keys, gives only table data equivalent to \"LITE\" plan type\n",
    "\n",
    "\n",
    "<br>\n",
    "Output objects detail below\n",
    "\n",
    "\n",
    "\n",
    "| Key Name \t| Parent \t| Type \t| Description \t| Availability \t|\n",
    "|-\t|-\t|-\t|-\t|-\t|\n",
    "| JobStatus \t| Job \t| String \t| Status of the triggered process \t| ALL Plans \t|\n",
    "| Pages \t| Job \t| Integer \t| Number of pages processed in the request \t| ALL Plans \t|\n",
    "| Tables \t| Job \t| Array \t| List of all tables found \t| ALL Plans \t|\n",
    "| Tables[0].Page \t| Table \t| Integer \t| Page number in which the table is found \t| ALL Plans \t|\n",
    "| Tables[0].CharacterConfidence \t| Table \t| Decimal \t| Accuracy of Characters recognized from the image \t| ALL Plans \t|\n",
    "| Tables[0].LayoutConfidence \t| Table \t| Decimal \t| Accuracy of table layout's design decision \t| ALL Plans \t|\n",
    "| Tables[0].TableJson \t| Table \t| Json/dict \t| Table Cell Text in key-value format with index orientation - {row#: {col#: }} \t| ALL Plans \t|\n",
    "| Tables[0].TableCoordinates \t| Table \t| Json/dict \t| Top-left & Bottom-right Cell Coordinates - {row#: {col#: }} \t| EXTRA Plan \t|\n",
    "| Tables[0].TableConfidence \t| Table \t| Json/dict \t| Cell level accuracy of detected characters - {row#: {col#: }} \t| EXTRA Plan \t|\n",
    "| Lines \t| Job \t| Array \t| List of page-wise lines text \t| FULL, EXTRA\t|\n",
    "| Lines[0].Page \t| Page \t| Integer \t| Page number in which the lines are found \t| Full Plan \t|\n",
    "| Lines[0].CharacterConfidence \t| Page \t| Decimal \t| Average Accuracy of all Characters recognized from the input-page \t| Full Plan \t|\n",
    "| Lines[0].LineArray \t| Page \t| Array \t| Ordered list of lines of the page \t|  \t|\n",
    "| Lines[0].LineArray[0].Line \t| Line \t| String \t| Detected text of the complete line \t| Full Plan \t|\n",
    "| Lines[0].LineArray[0].WordsArray \t| Line \t| Array \t| Word level datails in this line \t| EXTRA Plan \t|\n",
    "| Lines[0].LineArray[0].WordsArray[0].Conf \t| Word \t| Decimal \t| Accuracy of recognized characters of the word \t| EXTRA Plan \t|\n",
    "| Lines[0].LineArray[0].WordsArray[0].Word \t| Word \t| String \t| Detected text of the word \t| EXTRA Plan \t|\n",
    "| Lines[0].LineArray[0].WordsArray[0].Loc \t| Word \t| Array \t| Top-left & Bottom-right coordinates, w.r.t the input-page width-height dimensions \t| EXTRA Plan \t|"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 6. Make Corrections\n",
    "\n",
    "> **Objective**: To ease corrections on the most common issues with the `MakeCorrections` module.\n",
    "\n",
    "**Details:** The service relies on OCR (Optical Character Recognition) for character detection and deep learning models to detect tabular structures on the input. There may be a chance for merged rows or columns or incorrect type detections on low-quality inputs with a complex table layout or tightly packed columns. With those in mind, we want to offer the built-in service at the client-side to give control and ease in making corrections on the output. \n",
    "\n",
    "\n",
    "The module, `MakeCorrections`, currently supports below functionalities\n",
    "\n",
    "| Functionality        \t| Explanation                                    \t|\n",
    "|----------------------\t|------------------------------------------------\t|\n",
    "| Split Merged Rows    \t| Works well on cell values with no spaces       \t|\n",
    "| Split Merged Columns \t| Works well on cell values with no spaces       \t|\n",
    "| Fix Decimal Format   \t| To fix thousand and decimal separators         \t|\n",
    "| Fix Date Format      \t| To handle and modify incorrect date separators \t|\n",
    "| Fix Characters       \t| To fix incorrect character detections          \t|\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# First things first lets import the module and prepare for corrections\n",
    "\n",
    "from ExtractTable.common import MakeCorrections\n",
    "\n",
    "corrections = MakeCorrections(et_resp=et_sess.server_response)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.1 Split Merged Rows\n",
    "        \"\"\"\n",
    "        To split the merged rows into possible multiple rows\n",
    "        :return: reformatted list of dataframes\n",
    "        \"\"\"\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "corrected_table_dataframes = corrections.split_merged_rows()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.2 Split Merged Columns\n",
    "\n",
    "        \"\"\"\n",
    "        To split the merged columns into possible multiple columns\n",
    "        :param columns_idx: user preferred columns indices.\n",
    "                Default loops through all columns to find numeric or decimal columns\n",
    "        :param force_split: To force split through the columns\n",
    "        :return: reformatted list of dataframes\n",
    "        \"\"\"\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "corrected_table_dataframes = corrections.split_merged_columns()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.3 Fix Decimal Format\n",
    "\n",
    "\n",
    "        \"\"\"\n",
    "        To fix decimal and thousands separator values. Often commas as detected as period\n",
    "        :param columns_idx: user preferred columns indices.\n",
    "                Default loops through all columns to find numeric or decimal columns\n",
    "        :param decimal_separator: preferred decimal separator\n",
    "        :param thousands_separator: preferred thousands separator\n",
    "        :param decimal_position: preferred decimal position\n",
    "        :return: corrected list of dataframes\n",
    "        \"\"\"\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "corrected_table_dataframes = corrections.fix_decimal_format(decimal_separator=\".\", thousands_separator=\",\")"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.4 Fix Date Format\n",
    "\n",
    "        \"\"\"\n",
    "        To fix date formats of the column\n",
    "        Eg: 12|1212020 as 12/12/2020\n",
    "        :param columns_idx: user preferred columns indices.\n",
    "                Default loops through all columns to find Date Columns\n",
    "        :param delimiter: \"/\" or \"-\" whatelse you prefer\n",
    "        :return: correted list of dataframes\n",
    "        \"\"\"\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "corrected_table_dataframes = corrections.fix_date_format(delimiter=\"/\")"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.5 Fix Characters\n",
    "\n",
    "        \"\"\"\n",
    "        To replace incorrect character detections\n",
    "        Eg: $123,45.0I as $123,45.01\n",
    "        :param columns_idx: user preferred columns indices.\n",
    "                Default loops through all columns to find Date Columns\n",
    "        :param replace_ref: the replacement dictionary for reference\n",
    "                Eg: {\"I\": \"1\"}\n",
    "        :return: correted list of dataframes\n",
    "        \"\"\"\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "corrected_table_dataframes = corrections.fix_characters(self, columns_idx=[0, 1], replace_ref={\"I\": \"1\", \"S\": \"$\"})"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 7. Helpful Code Snippets\n",
    "\n",
    "Extra code snippets that are useful to perform some actions on the output. Based on the frequently asked questions."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 7.1 Get text data"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "# If your API Key supports \"Lines\"\n",
    "\n",
    "all_page_lines = []\n",
    "for each_page in et_sess.Lines:\n",
    "  for each_line in each_page['LinesArray']:\n",
    "    all_page_lines.append(each_line['Line'])\n",
    "  \n",
    "print(\"\\n\".join(all_page_lines))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 7.2 All tables output to a single excel"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "table_data = et_sess.process_file(filepath=Location_of_PDF_with_Tables, output_format=\"df\", pages=\"all\")\n",
    " \n",
    "import pandas as pd\n",
    "accumulate_all_dfs = pd.DataFrame()\n",
    "\n",
    "for each_df in table_data:\n",
    "    accumulate_all_dfs = accumulate_all_dfs.append(each_df, ignore_index=True)\n",
    "    # print(each_df.shape, accumulate_all_dfs.shape)\n",
    "\n",
    "print(\"Shape of all tables accumulated together is\", accumulate_all_dfs.shape)\n",
    "\n",
    "\n",
    "output_excel_location = <location_for_the_excel_output.xlsx>\n",
    "# Save the accumulated output to a single excel file\n",
    "accumulate_all_dfs.to_excel(output_excel_location, index=False, header=False)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 8. View transactions\n",
    "\n",
    "To view all the transactions that were triggered in the last 24 hours"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "et_sess.view_transactions()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Return a list of transactions, with each record having below properties\n",
    "\n",
    "|Property | Description |\n",
    "|---|---|\n",
    "|JobStatus | Status of the job|\n",
    "|Pages | number of pages of the input; can also be considered as number of credits consumed|\n",
    "|createdon | timestamp when the request was processed|\n",
    "|requested_filename | Filename received in the request|\n",
    "|txn_id | Unique identifier of the transaction, also referred as JobId to retrieve output via `get_result(JobId)`|"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# 9. Support & Contact\n",
    "\n",
    "Please do not hesitate to approach our developer team at pydevs@extracttable.com for any assitance needed or to report a bug"
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ]
}