New version of the interandroid language based on a core vocabulary (#5)

* start of vocab

* starting point vocabulary

* Display size of vocabulary

* move personality traits

* Basic creature

* Adds building

* separate phrasing

* Adds WordMeta with InfoMeta

* adds singletons

* base12 as text script

* Adds tests for info meta

* Adds test for syntax error in info meta

* conversion base12meta

* conversion both ways for Base12Meta

* Removes trailing underscore in test

* Adds validation tests for Base12Meta

* Tests with real vocabulary

* Tests double conversions

* ensure test for corruption

* tests for valid reference

* display duplicates

* Uses droid chars

* Inject core vocabulary

* fix spell mistakes

* supports multiple languages

* adds attribution

* Parse description md

* adds support for attribution in elm

* Adds pasring of media content

* Adds vocabulary description

* Adds Vocabulary translation

* Word composition 5 level

* Adds advanced example for word composition

* smarter dealing with seps

* description is jsut a string

* better reference model

* Adds phrasing

* Uses loadWordInfo

* Phrasing english

* translation in english

* fictional to classic conversion

* better core vocab

* basic determiner

* check for constant

* Validate purpose

* organize vocabulary by theme

* better random

* create random example

* support startPath

* find duplicates

* more adverbs

* Adds interandroid vscode support

* basic syntax highlight

* syntax coloration for core micro vocabulary

* make syntax color

* all colored

* units

* shape

* list management

* Adds chemistry

* sort alpha

* Adds checklist

* Adds universe

* Adds pattern

* fact checking

* wip

* refactoring with python naming convention

* small changes

* interactive code

* Adds vocabulary

* Adds a few words

* Adds words

* Adds more vocabulary

* generate vocab

* removes a few duplicates

* a bit of tidy up

* Adds applicative

* wip

* First round

* First working draft

* Loads all tasks in one go

* calculate word info

* Basic translation

* Adds translation back

* Adds word type

* a few changes

* Adds onhover

* Tweaks highlight

* Adds origin line

* good night

* Adds checkOrigin

* Adds word type

* Adds bottom footer

* Adds button and headers

* Adds header with sentence

* tweak css

* tweaks ui

* Adds phonetic

* Fixes origin

* Adds pronoun

* Adds mendeleev elements

* Updates readme

* build

.gitignore

@@ -24,3 +24,142 @@ Temporary Items
 
 temp/
 elm-stuff
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/

.vscode/settings.json

@@ -0,0 +1,4 @@
+{
+    "python.linting.pylintEnabled": true,
+    "python.linting.enabled": true
+}

Checklist.md

@@ -0,0 +1,32 @@
+# Checklists
+
+## Core vocabulary checklist
+
+The core vocabulary should be a limited set of words on which a wider language lexicon is been built. The basic premise is that there should be a reasonably short list of words to learn and memorize. Obviously there is a balance to strike between expressivity and size. The core language should be abstract and ambiguous. The language assumes a context that would clarify some ambiguity (for instance on Earth the time unit may be second).
+
+Words are organized in sequence of up to 12 words.
+
+Checklist for a sequence of 12 words:
+
+- [ ] The sequence should be thematic.
+- [ ] The order of these words is coherent, and follow one of these patterns: 
+    - Incremental: the first word is related to 0 or 1 and the last word to 11 or 12.
+    - Alphabetic order of the sequence words.
+    - A list of pairs (ex: uneasy easy), where the two parts of the pair are antonyms. The first word of the pair should represent negative (0) aspect while second word the positive (1) aspect. Whenever possible the second word (positive) should be used to sort the tuples alphabetically. (negative words often starts with same prefix and make them a bad fit for sorting).
+- [ ] The sequence is reasonably easy to remember.
+- [ ] The sequence helps to narrow down a meaning.
+
+
+Checklist for a word in the core vocabulary:
+- [ ] The word is abstract enough.
+- [ ] The word is not specific to Earth and would be relevant for most planets. For example, not every planet would have tigers or Japan, but they could have mammals and countries.
+- [ ] The word is useful for adding meaning to other words the same way a prefix or suffix in English would be.
+- [ ] The word is a known mathematical, physical, computing concept that would always be relevant.
+- [ ] The word could be poetically used for similar concepts in different fields.
+- [ ] Synonyms in the core vocabulary should be avoided unless they play a role in a sequence.
+- [ ] The word should be frequently used in the resulting language. A niche word should be discarded if it could be easily replaced by another word or combination of words in the core vocabulary.
+- [ ] The word does not have a more common synonym.
+
+
+
+

History.md

@@ -0,0 +1,10 @@
+# History
+
+## Version 0.5-alpha
+
+The second version is a complete rewrite of the first version. The new language is based on a lower language called the core-vocabulary that has a limited number of words. Words are created using this core-vocabulary and registered in a dictionary. English is the initial language, but the codebase is ready to support additional languages.
+
+## Version 0.3-alpha
+
+This first version of the interandroid language had almost three thousand words picked up manually from diverse open data sets. In order to facilitate eventual memorisation (by humans), often the interandroid name will be based on the phonetic of the word, usually in English. Names with similar pronunciation will conflict. The vocabulary was not really practical or easy to remember. The editor allowed translation between English and interandroid and vice-versa. In addition, you would have access to the phonetic of the whole sentence. 
+

Makefile

@@ -5,23 +5,32 @@ BUILD = docs
 HTML = elm/html
 
 vocabulary:
-	python tools/inject-vocabulary.py
+	python tools/inject-core-vocabulary.py
+	python tools/inject-core-vocabulary-media.py
+	python tools/inject-core-vocabulary-description.py
+	cat data/en/core-vocabulary-translation*.csv > docs/data/en/core-vocabulary-translation.csv
 
 build: build-directory vocabulary html js
 
+org:
+	python tools/organize-vocabulary.py
+
 build-directory:
 	mkdir -p $(BUILD)
 
 html:
 	cp $(HTML)/index.html $(BUILD)/index.html
 js:
-	cd elm; elm make $(SRC)/App.elm --output ../$(BUILD)/app.js
+	cd elm; elm make $(SRC)/Main.elm --output ../$(BUILD)/main.js
 
 start:
-	open http://localhost:7000; cd docs;python -m SimpleHTTPServer 7000
+	open -n -a "Google Chrome" --args --incognito http://localhost:7000; cd docs;http-server -p 7000
 
 beautify:
 	cd elm; elm-format src/ --yes
 
 test:
 	cd elm; elm-test
+
+syntax-color:
+	ln -s "`pwd`/vscode/interandroid" ~/.vscode/extensions/interandroid-0.0.1

README.md

@@ -4,11 +4,50 @@
 
 ## Translation
 
- https://barking-up-the-binary-tree.github.io/interandroid-fictional-language/
+The interandroid space language is mainly used for communication between [Android robots](https://en.wikipedia.org/wiki/Android_(robot)) in distant worlds.
 
+The language itself is a work in progress, but feel free to use it for any sort of creative writings (poems, songs, short-stories ...). If you do so, we will be delighted to hear about it ...
+
+[Here](https://barking-up-the-binary-tree.github.io/interandroid-fictional-language/) you can find a basic translator.
+
+We are building a [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) dictionary, and you are much welcome to contribute to it. Words in that dictionary are based on a [core-vocabulary](data/en/core-vocabulary.iandroid.txt) (origin) that we will try to keep stable - as soon as we have a clear view of what words should really be in it -.
+
+The language is very classic in that it supports nouns, verbs, adjectives, adverbs, pronouns, prepositions, conjunctions, and determiners. The first letter of the word gives a hint about this.
+
+Because of the core-vocabulary, direct translation between an english word and an interandroid one is not always possible. In some cases, the interandroid language may be more precise (ex: pronouns) and in some other cases vaguer (ex: animals). Animals for instance, are only vaguely related to their Earth's counterparts. A tiger and a lion may be represented by the same word. You have to keep in mind, that in the remote worlds, well known Earth personalities and animals may not exist at all. For this reason, the core language tends to contain only word that one may expect to find in any world.
+
+The alphabet for interandroid words is made of the twelve characters: '┼', '╀', '┾', '╄', '╁', '╂', '╆', '╊', '┽', '╃', '┿', '╇'. For composed words, the separator '┄' can be used, unfortunately this is not yet supported in the UI.
+
+In addition, you can separate:
+* words with ◌,
+* clauses with ◌◌,
+* sentence clauses with ◌◌◌,
+* sentences with ◌◌◌◌,
+* paragraphs with ◌◌◌◌◌.
+
+The language supports the following tenses:
+* FarPast, a long time ago. It may be a few hundreds years ago or millions of years ago.
+* Past, recent past, such as last week.
+* Present, the time now.
+* Future, the close future, such as tomorrow or next month.
+* FarFuture, in a very long time, probably several generations later or in a few centuries.
+* Hypothesis, statement that could possibly happen.
+* Request, a call for help in satisfying a request.
+* Promise, a pledge to satisfy a particular request.
+* Attempt, an attempt at implementing a proposition.
+* Prediction, the forecasting of a future event.
+* Warning, a warning about something that could happen.
+* Fantasy, a fantasy of what could happen.
+
+The exact grammar is not formalised yet, but the structure of a sentence is expected to respect the classic "subject verb object".
+The assumption is that the clause before the verb(s) is the subject while the clause after is the object. Before the subject, it should be possible to add clauses to indicate a time, location or any other context.
+
+While the language may sound rich enough for most situations, its ultimate purpose is to be used by fictional (or perhaps real) Android chat bots. They may not all be as talented as [C-3PO](https://en.wikipedia.org/wiki/C-3PO) to understand complex language construction. For this reason, the grammar should be predictable and each sentence should try to limit the number of ideas. Even humans struggle with very long sentences, and primitive bots should not be any different. On the other hand, too short sentences may feel unatural and dull, so a balance will have to be striked.
+
+That's it! Best of luck !!
 
 ## Licenses
 
-* Copyright (c) 2017-2019 Olivier Huin
+* Copyright (c) 2017-2020 Olivier Huin
 * The source code of this program (Elm, python) is under The MIT License
 * The language and the use of the language itself will have very open license that still need to be discussed, most likely something in the spirit of public domain https://creativecommons.org/share-your-work/public-domain/cc0/

data/en/attribution.json

@@ -0,0 +1,9 @@
+[
+    {
+        "id": "attrib:olih/by-sa",
+        "attributionName": "Olivier Huin",
+        "attributionURL": "https://github.com/olih",
+        "licenseName": "Creative Commons Attribution-ShareAlike 4.0 International License",
+        "licenseURL": "https://creativecommons.org/licenses/by-sa/4.0/"
+    }
+]