gramtool - a Python library for morphological analysis.
Main purpose of this library is to identify given phrase and provide all information about that phrase, its grammatical category, part of speech, used affixes and etc.
gramtool requires Python 3 version.
On Ubuntu 14.04:
$ sudo apt-get install python-dev libhunspell-dev $ pip install gramtool
$ gramtool žodžiuose žodžiuose [nmpl] -> žodis žodžiuose [nmsl] -> žodžiai Masculine dis (medis) [nmsn] žodis [nmsg] žodžio [nmsd] žodžiui [nmsa] žodį [nmsi] žodžiu [nmsl] žodyje [nmpn] žodžiai [nmpg] žodžių [nmpd] žodžiams [nmpa] žodžius [nmpi] žodžiais [nmpl] žodžiuose AI Miltai only plural [nmsn] žodžiai [nmsg] žodžių [nmsd] žodžiams [nmsa] žodžius [nmsi] žodžiais [nmsl] žodžiuose $ gramtool žmogus --case=locative žmoguje
Find word lemma:
assert gramtool.get_lemma('žodžiai') == 'žodis'Get different grammatical form:
assert gramtool.change_form('dėžė', case='locative') == 'dėžėje'gramtool uses Hunspell dictionaries and grammar files to define grammars for each language.
For a language these files are used:
grammar.yaml- Defines grammatical category.
grammar- Defines morphological structure, using
grammar.yaml. hunspell.aff,hunspell.dic- Dictionaries, that defines all possible correct word forms.
gramtool takes a word, then tries to find best match from grammar file,
identifying all possible word form variants. Then using Hunspell spell checker,
for each matching grammar rule, all forms are generated. Only those rules, that
passes spell checking for all generated forms are considered as correct rules
for that given word.
To define new grammar for a language, first you need to define all grammar
parts in grammar.yaml file. All defined symbols must be compatible with
general, language independed grammar.yaml file. General grammar.yaml is
used to connect all defined rules between all languages, for example, Noun,
should be known as Noun in all languages.
Second thing, you need to do is describing morphological structure. This you
need to put in grammar file.
Here is a simples grammar file example:
@rule nouns ns . . np . s
This file defines english noun. @rule nouns opens new rule, all lines
below, untile next rule or macro definition belongs to this rule.
Second line from example above:
ns . .
Tells, that n - noun, s - singulare, . - does not have any prefixes
and . - does not have any suffixes.
Third line:
np . s
Tells that plural form of a noun must be appended with s suffix.
So in as you probably understood, each rule line consists of:
- grammatical form specification, using symbols taken from
grammar.yamlfile, I will call in gram-spec - prefix part
- suffix part.
Lets take a look at an example with english verbs:
@macro verb xs1 . . # I xs2 . . # you xs3 . . # he xp1 . . # we xp2 . . # you xp3 . . # they @macro verb-s xs1 . . # I xs2 . . # you xs3 . s # he xp1 . . # we xp2 . . # you xp3 . . # they @macro verb-am xs1 am+ . # I xs2 are+ . # you xs3 is+ . # he xp1 are+ . # we xp2 are+ . # you xp3 are+ . # they @macro verb-was xs1 was+ . # I xs2 where+ . # you xs3 was+ . # he xp1 where+ . # we xp2 where+ . # you xp3 where+ . # they @macro verb-have xs1 have+ . # I xs2 have+ . # you xs3 has+ . # he xp1 have+ . # we xp2 have+ . # you xp3 have+ . # they @rule regular-verbs + verb-s ***p . . + verb-am ***pc . ing + verb ***ss . ed + verb-was ***sc . ing + verb-have ***pp . ed + verb-have ***ppc been+ . + verb ***sp had+ ed + verb ***spc had+been+ ed + verb ***f will+ . + verb ***fc will+be+ ing + verb ***fp will+have+ ed + verb ***p-C would+ .
Here we have many new things. First of all we see five macros: verb,
verb-s, verb-am, verb-was, verb-have. Macros are used to be
included into other rules. Also it is possible to include macro into macro,
rule into rule.
Here is example, how a macro is included:
+ verb-s ***p . .
In this example, macro verb-s will be included into rule regular-verbs.
It means, that all lines, defined in verb-s, will be included into
regular-verbs rule. Also, ***p parameter specifies, that during
inclusion, all gram-specs from verb-s will be replaced with forth letter to
become p.
Also, when including, specified prefixes and suffixes will be prepended with specified affixes in inclusion parameters.
You can include not only specified macro or rule, by name, but it is also possible to include same rule again or parent rule from a macro. Also it is possible to specify a filter, that tells what lines will be included.
Starting a rule or macro:
@rule <name>- Start new rule.
@macro <name>- Start new macro. Macros will net be used when generating word forms, macros can only be included into other rules.
Both, rules and macros can contain same lines, specifying possible word forms. Word form line can be defined in these forms:
<spec> <stem>- This form is used to define irregular word forms, when stem is not same for all other forms, for example, words go and went have different stems.
<spec> <prefix> <suffix>- This form is used to define regular word forms.
Both, rules and macros can contain includes:
+[<level>] <name>Simple include form. Just includes all lines from rule or macro named with
<name>.<name>has several special symbols:.- include self lines.@- include top most rule lines.Optional level is a number that restricts included lines to only those with lower inclusion level. Inclusion level is assigned to each line when inclusio is performed. Each line after inclusion has level as specified in
+[<level>].+[<level>] <name> <spec> <prefix> <suffix>- Same as above, but all included lines will be extended with
<spec>and specified<prefix>and<suffix>will be prepended to prefixes and suffixes of included line. +[<level>] <name> <spec> <prefix> <suffix> <filter>- Same as above, but will be included only lines, whose
<spec>will match specified<filter>.