README_PLUGINS

-------------------------------------------------------------------------------
---  HIGHLIGHT PLUGIN MANUAL - Version 3.40   --------------- November 2017 ---
-------------------------------------------------------------------------------

CONTENT
-------------------------------------------------------------------------------

1. ABOUT
2. SCRIPT STRUCTURE
3. SYNTAX CHUNK ELEMENTS
   3.1 FUNCTION ADDKEYWORD
   3.2 FUNCTION ONSTATECHANGE
   3.3 DECORATE FUNCTIONS
4. THEME CHUNK ELEMENTS
5. STEP BY STEP
6. EXAMPLES
7. PLUG-IN USAGE


1. ABOUT
-------------------------------------------------------------------------------

The plug-in interface allows modifications of syntax parsing and colouring.
The output's header or footer can be enhanced, and recognized syntax elements
may be outputted with additional information.
A common task would be to define a new set of syntax keywords, or to add items
to existing keyword groups. More advanced plug-ins add tooltips based on ctags 
input or source code folding to the output (see section 6).


2. SCRIPT STRUCTURE
-------------------------------------------------------------------------------

The following script contains the basic plug-in structure which has no effect 
on the output:

Description="Plugin Boilerplate"

-- optional parameter: syntax description
function syntaxUpdate(desc)
end

-- optional parameter: theme description
function themeUpdate(desc)
end

Plugins={
  { Type="theme", Chunk=themeUpdate },
  { Type="lang", Chunk=syntaxUpdate },
}

The first line contains a description which gives a short summary of the
plug-in effects.

The next lines contain function definitions:
The syntaxUpdate function is applied on syntax definition scripts (*.lang),
whereas the themeUpdate function is applied on colour themes (*.theme).
The desc parameter contains the description string of the loaded syntax
definition or colour theme. This information can be used to restrict
modifications to certain kinds of input (i.e. only C code should be
enhanced with syslog keywords).

Finally the Plugins array connects the functions to the Lua states which 
are created when highlight loads a lang or theme script. In this example,
themeUpdate is connected to the theme state, and syntaxUpdate to the lang
state. It is not required to always define both connections if your plugin
only affects one of the config script types.


3. SYNTAX CHUNK ELEMENTS
-------------------------------------------------------------------------------

The following list includes all variables and constants you may use to adjust 
the parser's syntax highlighting.

Syntax definition items:

Comments: table
Description: string
Digits: string
EnableIndentation: boolean
Identifiers: string
IgnoreCase: boolean
Keywords: table
NestedSections: table
Operators: string
PreProcessor: table
Strings: table

Document modification items:

HeaderInjection: string
FooterInjection: string

Read only (internal highlighting states):

HL_STANDARD: number
HL_BLOCK_COMMENT: number
HL_BLOCK_COMMENT_END: number
HL_EMBEDDED_CODE_BEGIN: number
HL_EMBEDDED_CODE_END: number
HL_ESC_SEQ: number
HL_ESC_SEQ_END: number
HL_IDENTIFIER_BEGIN: number
HL_IDENTIFIER_END: number
HL_KEYWORD: number
HL_KEYWORD_END: number
HL_LINENUMBER: number
HL_LINE_COMMENT: number
HL_LINE_COMMENT_END: number
HL_NUMBER: number
HL_OPERATOR: number
HL_OPERATOR_END: number
HL_PREPROC: number
HL_PREPROC_END: number
HL_PREPROC_STRING: number
HL_STRING: number
HL_STRING_END: number
HL_UNKNOWN: number
HL_REJECT: number

Read only (output document format):

HL_OUTPUT: number (selected format)
HL_FORMAT_HTML: number
HL_FORMAT_XHTML: number
HL_FORMAT_TEX: number
HL_FORMAT_LATEX: number
HL_FORMAT_RTF: number
HL_FORMAT_ANSI: number
HL_FORMAT_XTERM256: number
HL_FORMAT_SVG: number
HL_FORMAT_BBCODE: number
HL_FORMAT_PANGO: number
HL_FORMAT_ODT: number

Read only (other):

HL_PLUGIN_PARAM: string (set with --plug-in-param)
HL_LANG_DIR: string (path of language definition directory)

Functions:

AddKeyword: function
OnStateChange: function
Decorate: function
DecorateLineBegin: function
DecorateLineEnd: function

IMPORTANT: Functions will only be executed if they are defined as local
functions within the "lang" chunk referenced in the Plugins array. 
They will be ignored when defined elsewhere in the script.


3.1 FUNCTION ADDKEYWORD
-------------------------------------------------------------------------------

This function will add a keyword to one of the the internal keyword lists.
It has no effect if the keyword was added before.
Keywords added with AddKeyword will remain active for all files of the same
syntax if highlight is in batch mode.

AddKeyword(keyword, kwGroupID)

  Parameters: keyword:   string which should be added to a keyword list
              kwGroupID: keyword group ID of the keyword
  Returns:    true if successfull
  

3.2 FUNCTION ONSTATECHANGE
-------------------------------------------------------------------------------

As this function is more commonly used in language definitions, refer to README
for its detailed description.

Example for its usage in a plugin context:

function OnStateChange(oldState, newState, token, kwgroup)
   if newState==HL_KEYWORD and kwgroup==5 then
      AddKeyword(token, 5)
   end
   return newState
end

This function adds the current keyword to the internal keyword list if the 
keyword belongs to keyword group 5. If keyword group 5 is defined by a regex,
this token will be recognized later as keyword even if the regular expression 
does no longer match.


3.3 DECORATE FUNCTIONS
-------------------------------------------------------------------------------

The Decorate function is a hook which is called if a syntax token has been 
identified. It can be used to alter the token or to add additional text in the
target output format (e.g. hyperlinks). 

Decorate(token, state, kwGroupID, stateTrace)

  Hook Event: Token identification
  Parameters: token:      current token
              state:      current state
              kwGroupID:  if state is HL_KEYWORD, the parameter
                          contains the keyword group ID
              stateTrace: string containing past states of the current line;
                          separated by ';'; limited to 100 entries
  Returns:    Altered token string or nothing if original token should be 
              outputted

Examples:

function Decorate(token, state)
  if (state == HL_KEYWORD) then
    return string.upper(token)
  end
end

This function converts all keywords to upper case.


The functions DecorateLineBegin and DecorateLineEnd are called if a new line
starts or ends. They can be used to add special formatting to lines of code.

DecorateLineBegin(lineNumber)

  Hook Event: output of a new line
  Parameters: lineNumber: the current line number
  Returns:    A string to be prepended to a new line (or nothing) 

DecorateLineEnd(lineNumber)

  Hook Event: output of a line ending
  Parameters: lineNumber: the current line number
  Returns:    A string to be appended to a line (or nothing)
  

IMPORTANT: The return value of Decorate functions will be embedded in the 
formatting tags of the output format. The return values are not modified or 
validated by highlight.

  
4. THEME CHUNK ELEMENTS
-------------------------------------------------------------------------------

The following list includes all items you may overwrite or extend to adjust
the formatting (colour and font attributes) of the output:

Output formatting items:

Default: table
Canvas: table
Number: table
Escape: table
String: table
StringPreProc: table
BlockComment: table
PreProcessor: table
LineNum: table
Operator: table
LineComment: table
Keywords: table

Read only (output document format)::

HL_OUTPUT: number
HL_FORMAT_HTML: number
HL_FORMAT_XHTML: number
HL_FORMAT_TEX: number
HL_FORMAT_LATEX: number
HL_FORMAT_RTF: number
HL_FORMAT_ANSI: number
HL_FORMAT_XTERM256: number
HL_FORMAT_SVG: number
HL_FORMAT_BBCODE: number
HL_FORMAT_PANGO: number
HL_FORMAT_ODT: number

Add additional stying information:

Injections: table


5. STEP BY STEP
-------------------------------------------------------------------------------

This example will add reference hyperlinks to Qt keywords:

-- first add a description of what the plug-in does
Description="Add qtproject.org reference links to HTML, LaTeX or RTF output"

-- the syntaxUpdate function contains code related to syntax recognition
function syntaxUpdate(desc)

  -- if the current file is no C++ file we exit
  if desc~="C and C++" then
     return
  end
      
  -- this function returns a qt-project reference link of the given token
  function getURL(token)
     -- generate the URL
     url='http://qt-project.org/doc/qt-4.8/'..string.lower(token).. '.html'
     
     -- embed the URL in a hyperlink according to the output format
     -- first HTML, then LaTeX and RTF
     if (HL_OUTPUT== HL_FORMAT_HTML or HL_OUTPUT == HL_FORMAT_XHTML) then
         return '<a class="hl" target="new" href="' 
                .. url .. '">'.. token .. '</a>'
     elseif (HL_OUTPUT == HL_FORMAT_LATEX) then
         return '\\href{'..url..'}{'..token..'}'
     elseif (HL_OUTPUT == HL_FORMAT_RTF) then
         return '{{\\field{\\*\\fldinst HYPERLINK "'
                ..url..'" }{\\fldrslt\\ul\\ulc0 '..token..'}}}'
     end
   end

  -- the Decorate function will be invoked for every recognized token
  function Decorate(token, state)

    -- we are only interested in keywords, preprocessor or default items
    if (state ~= HL_STANDARD and state ~= HL_KEYWORD and 
        state ~=HL_PREPROC) then
      return
    end

    -- Qt keywords start with Q, followed by an upper and a lower case letter
    -- if this pattern applies to the token, we return the URL
    -- if we return nothing, the token is outputted as is
    if string.find(token, "Q%u%l")==1 then
      return getURL(token)
    end

  end
end

-- the themeUpdate function contains code related to the theme
function themeUpdate(desc)
  -- the Injections table can be used to add style information to the theme
  
  -- HTML: we add additional CSS style information to beautify hyperlinks,
  -- they should have the same color as their surrounding tags
  if (HL_OUTPUT == HL_FORMAT_HTML or HL_OUTPUT == HL_FORMAT_XHTML) then
    Injections[#Injections+1]=
      "a.hl, a.hl:visited {color:inherit;font-weight:inherit;}"

  -- LaTeX: hyperlinks require the hyperref package, so we add this here
  -- the colorlinks and pdfborderstyle options remove ugly boxes in the output
  elseif (HL_OUTPUT==HL_FORMAT_LATEX) then
    Injections[#Injections+1]=
      "\\usepackage[colorlinks=false, pdfborderstyle={/S/U/W 1}]{hyperref}"
  end
end

-- let highlight load the chunks
Plugins={
  { Type="lang", Chunk=syntaxUpdate },
  { Type="theme", Chunk=themeUpdate },
}


6. EXAMPLES
-------------------------------------------------------------------------------

The plugins directory contains example scripts, including: 

bash_functions.lua
Description: Add function names to keyword list
Features:    Adds new keyword group based on a regex, defines OnStateChange,
             uses AddKeyword

theme_invert.lua
Description: Invert colours of the original theme
Features:    Modifies all color attributes of the theme script, uses Lua
             pattern matching
          
cpp_ref_cplusplus_com.lua
Description: Add qtproject.org reference links to HTML, LaTeX or RTF output of
             C++ code
Features:    Uses Decorate to add hyperlinks for a defined set of C++ keywords.
             Adds CSS styles with Injections.

ctags_html_tooltips.lua
Description: Add tooltips based on a ctags file (default input file: tags)
Features:    Uses file input (defined by cli option --plug-in-param) and
             parses tags data before Decorate is called.
             
outhtml_curly_brackets_matcher.lua
Description: Shows matching curly brackets in HTML output.
Features:    Uses Decorate to add span tags with unique ids to opening and 
             closing brackets.
             Adds JavaScript with HeaderInjection variable.
             Inserts additional CSS styles with Injections variable.
             
outhtml_keyword_matcher.lua
Description: Shows matching keywords in HTML output.
Features:    Uses Decorate to add span tags with unique ids to keywords.
             Uses OnStateChange to assign an internal ID to each keyword.
             Adds JavaScript with HeaderInjection variable.
             Inserts additional CSS styles with Injections variable.
             
outhtml_codefold.lua
Description: Adds code folding for C style languages, Pascal, Lua and Ruby to 
             HTML output
Features:    Uses DecorateLineBegin and DecorateLineEnd to add ID-spans to each 
             line.
             Applies Decorate to each code block delimiter to add onClick event 
             handlers.
             Adds JavaScript with HeaderInjection and FooterInjection variables.
             Inserts additional CSS styles with Injections variable.
             
 
7. PLUG-IN USAGE
-------------------------------------------------------------------------------

Command line interface:
Run highlight --list-scripts=plugins to show all installed plug-ins.

Use --plug-in to load a plug-in script file. This option can be applied 
more than once to apply several plug-ins. Omit the .lua suffix.
You can store your plug-in scripts for testing in ~/.highlight/plugins.

Example:
highlight my.cpp -Ilz --plug-in=html_curly_brackets_matcher > ~/test_out/my.html


GUI:
Add the plug-in scripts in the plug-in selection tab and enable them using the
checkboxes.