Applanga Command Line Interface Version 1.0.105

CHANGELOG.md

@@ -5,7 +5,12 @@
 *Applanga CLI Documentation:* <https://www.applanga.com/docs-integration/cli>
 ***
 
-### Version 1.0.104 (19 Dec 2024)
+### Version 1.0.105 (20 Feb 2025)
+#### Added
+- Added `csv`, `tsv` and `xls` file formats
+---
+
+### Version 1.0.104 (19 Dez 2024)
 #### Fixed
 - Added improved contradictory configuration error 
 - Fixed init script for windows file path conversion

README.md

@@ -1,7 +1,7 @@
 # Applanga Localization Command Line Interface (CLI)
 
 ***
-*Version:* 1.0.104
+*Version:* 1.0.105
 
 *Website:* <https://www.applanga.com>
 
@@ -193,6 +193,7 @@ There are a few mandatory and several optional properties that you can use to cu
 	 - android_xml : [Android XML ](https://www.applanga.com/docs/formats/android_xml) (.xml)
 	 - angular_translate_json : [Angular Translate](https://www.applanga.com/docs/formats/angular_translate_json) (.json)
 	 - chrome_i18n_json : [Chrome i18n](https://www.applanga.com/docs/formats/chrome_i18n_json) (.json)
+	 - csv : [CSV](https://www.applanga.com/docs/formats/csv) (.csv)
 	 - ember_i18n_json_module : [Ember i18n JSON Module](https://www.applanga.com/docs/formats/ember_i18n_json_module) (.js)
 	 - ini : [INI](https://www.applanga.com/docs/formats/ini) (.ini)
 	 - gettext_po : [Gettext PO File](https://www.applanga.com/docs/formats/gettext_po) (.po)
@@ -217,7 +218,9 @@ There are a few mandatory and several optional properties that you can use to cu
 	 - microsoft_resw: [Microsoft Resw](https://www.applanga.com/docs/formats/microsoft_resx) (.resw)
 	 - microsoft_resx: [Microsoft Resx](https://www.applanga.com/docs/formats/microsoft_resx) (.resx)
 	 - toml : [Toml](https://www.applanga.com/docs/formats/toml) (.toml)
+	 - tsv : [TSV](https://www.applanga.com/docs/formats/tsv) (.tsv)
 	 - xliff: [Xliff](https://www.applanga.com/docs/formats/xliff) (.xliff)
+	 - xls: [Microsoft Excel](https://www.applanga.com/docs/formats/xls) (.xls or .xlsx)
 
 	***Example:*** `"file_format": "android_xml"`
 
@@ -419,21 +422,77 @@ It is possible to set the variable `<language>` in the path. In the "source" blo
 
 	***Example:*** `"remove_cr_char": true`
 
+- **"includeStatus"** *(pull commands only)*
+
+	The option is applicable only to file formats: csv, tsv, xls and xliff. If the value is set to `true`, the downloded files will contain the current statuse and the lastest comment for a particular string. This options by default is set to `false`.
+
+	***Example:*** `"includeStatus": true`
+
+- **"excludeBaseLang"** *(pull commands only)*
+
+	The option is applicable only to file formats: csv, tsv, and xls. If the value is set to `true`, the downloaded files do not containe the base language column. This options by default is set to `false`.
+
+	***Example:*** `"excludeBaseLang": true`
+
+- **"excludeHeaderRow"**
+
+	The option is applicable only to file formats: csv, tsv, and xls. This options by default is set to `false`.
+
+	If the value is set to `true` for pull command, the downloaded file does not containe the usual header row that describes what is in which column (e.g. ID, en, description, etc). Instead, the exported content should start in row 1 without the header row. 
+
+	For pull command, for the `true` value the translation data is read from file from the first raw (there is no header). Otherwise the first row is not included in scope and treated as a header. 
+
+	***Example:*** `"excludeHeaderRow": true`
+
+- **"columnDescription"**  *(push commands only, **mandatory for csv, tsv, and xls**)* 
+
+	The option is applicable only to file formats: csv, tsv, and xls. For these file formats the option must be provided. or the import will fail. It should contain an object linking columns numbers to the type of data found in them. The columns are numberd from 0 (A -> 0, B -> 1, etc.). The data types are:
+
+	- entry `KEY`
+	- exact language code (i.e. `en`) or language placeholder `<language>`
+	- entry `DESCRIPTION`
+	- string max `LENGTH`
+	- `METADATA_` keyword followed by the metadata name
+
+	A minimum the `KEY` and any other column must be specified.
+
+	***Example:*** 
+    ```json
+    "columnDescription": {
+        "KEY": 0,
+        "da": 2,
+        "DESCRIPTION": 4,
+        "LENGTH": 5,
+        "METADATA_product name": 6,
+        "METADATA_project": 7
+    }
+    ```
+
+- **sheetName**  *(push commands only)*
+
+	The option is applicable only to xls file formats. For multi-sheet Excel files, the data is imported only from one sheet. The option needs to be given for the data to be imported from a specific sheet. Otherwise by defualt the first sheet is imported.
+
+	***Example:*** `"sheetName": "Latest Sources"`
+
 
 ### Xliff specific options
 
 The following options will only work if the file_format is set to `xliff` (see [Applanga Xliff Format Documentation](https://applanga.com/docs/formats/xliff) for more information of the xliff format).
 
 
-- **"xliffStatus"**
+- **"xliffStatus"**  *- deprecated*
 
-	The files will import/export the given statuses of the xliff file. This options by default is set to `false`.
+	Use **importStatus** option for push commands or **includeStatus** in pull commands
+
+- **"importStatus"** *(push commands only)*
+
+	The statuses in the xliff file will be imported into Applanga. This options by default is set to  `false`.
 
-	***Example:*** `"xliffStatus": true`
+	***Example:*** `"importStatus": true`
 
 - **"createUnknownCustomStates"** *(push commands only)*
 
-	Default value is `false`. If set to true custom statuses provided inside the xliff format will be importet into Applanga.
+	Default value is `false`. If set to true custom statuses provided inside the xliff format will be imported into Applanga. The option only works if **importStatus** is also set to `true`.
 
 	***Example:*** `"createUnknownCustomStates": true`
 

commands/init.py

@@ -39,8 +39,8 @@ def init(ctx):
     target_path = None
     baselanguage_path = None
     tag = None
-	
-	
+    keyCol = None
+    langCol = None
 
     # Request all properties which are needed and did not get supplied
     # as arguments by user
@@ -111,6 +111,21 @@ def init(ctx):
         tag = input('Tag name [\"%s\"]: ' % default_tag_name)
         tag = tag or default_tag_name
 
+    if file_format in ['csv', 'tsv', 'xls']:
+        keyCol = input('\nChoose \'KEY\' column number (A -> 0, B -> 1, etc.) [\"0\"]:')
+        try:
+            keyCol = int(keyCol)
+        except:
+            keyCol = 0
+        if keyCol < 0:
+            keyCol = 0
+        langCol = input('\nChoose \'%s\' base language column number (A -> 0, B -> 1, etc.) [\"%d\"]:' % (base_language, keyCol + 1))
+        try:
+            langCol = int(langCol)
+        except:
+            langCol = keyCol + 1
+        if langCol < 0 or langCol == keyCol:
+            langCol = keyCol + 1
 
     # Save the configuration file
     configfile_data = {
@@ -187,5 +202,12 @@ def init(ctx):
             }
         }
 
+    #add mandatory minimum for spreadsheet formats
+    if file_format in ['csv', 'tsv', 'xls']:
+        configfile_data['app']['push']['source'][0]['columnDescription'] = {
+            'KEY': keyCol,
+            base_language: langCol
+        }
+
     # Write the new config file to local folder
     config_file.write(configfile_data)

lib/api.py

@@ -57,16 +57,23 @@ def downloadFile(file_data, debug=False):
     if file_data['file_format'] in ['nested_json', 'react_nested_json', 'ruby_on_rails_yaml', 'symfony_yaml', 'symfony2_yaml', 'ember_i18n_json_module', 'node_2_json', 'go_i18n_json'] and 'disable_plurals' in file_data:
             request_options['disablePlurals'] = file_data['disable_plurals'] is True
 
-    if 'includeMetadata' in file_data:
+    if 'includeMetadata' in file_data and file_data['file_format'] in ['csv', 'tsv', 'xliff', 'xls', 'arb']:
         request_options['includeMetadata'] = file_data['includeMetadata']
 
     if 'includeInvisibleId' in file_data:
         request_options['includeInvisibleId'] = file_data['includeInvisibleId']
 
-    if 'xliffStatus' in file_data:
-        request_options['xliffStatus'] = file_data['xliffStatus']
+    if 'xliffStatus' in file_data and file_data['file_format'] in ['xliff']:
+        #'includeStatus' replaces 'xliffStatus' so both options should not be present
+        if 'includeStatus' in file_data:
+            raise ApplangaRequestException('The \'includeStatus\' option replaced \'xliffStatus\' options. Please use \'includeStatus\' option.\nFor more informations and examples on how todo that please refer to the Applanga CLI Integration Documentation.')
+        request_options['includeStatus'] = file_data['xliffStatus']
 
-    if 'includeContextUrl' in file_data:
+    #'includeStatus' overwrites 'xliffStatus' if both are present
+    if 'includeStatus' in file_data and file_data['file_format'] in ['xliff', 'csv', 'tsv', 'xls']:
+        request_options['includeStatus'] = file_data['includeStatus']
+
+    if 'includeContextUrl' in file_data and file_data['file_format'] in ['xliff']:
         request_options['includeContextUrl'] = file_data['includeContextUrl']
 
     # check conditions for key_prefix
@@ -87,6 +94,14 @@ def downloadFile(file_data, debug=False):
         if file_data['remove_cr_char'] == True:
             request_options['removeCrChar'] = True
 
+    if file_data['file_format'] in ['csv', 'tsv', 'xls']:
+        if 'excludeHeaderRow' in file_data:
+            request_options['excludeHeaderRow'] = file_data['excludeHeaderRow']
+
+        if 'excludeBaseLang' in file_data:
+            request_options['excludeBaseLang'] = file_data['excludeBaseLang']
+
+
     try:
         # Request the file from server
         request_data = {
@@ -214,6 +229,51 @@ def uploadFiles(upload_files, force=False, draft=False, debug=False):
                 )
                 continue
 
+        #extra checks for comuplosry properties for Sheet imports
+        if source['file_format'] in ['csv', 'tsv', 'xls']:
+            #does 'columnDescription' property exist?
+            if 'columnDescription' not in source:
+                return_data.append( {
+                    'path': source['file_format'],
+                    'error': 'For \'csv\', \'tsv\' and \'xls\' providing the \'columnDescription\' property is compulsory. \nFor more informations and examples on how todo that please refer to the Applanga CLI Integration Documentation.'
+                })
+                continue
+
+            columnDescription = source['columnDescription']
+            #does it contan 'KEY' property
+            if ('KEY' not in columnDescription):
+               return_data.append({
+                    'path': source['columnDescription'],
+                    'error': 'For \'csv\', \'tsv\' and \'xls\' providing the \'KEY\' in \'columnDescription\' property is compulsory. \nFor more informations and examples on how todo that please refer to the Applanga CLI Integration Documentation.'
+                })
+               continue
+
+
+            valid_properties = ['<language>', 'DESCRIPTION', 'LENGTH']
+            language_regex = re.compile('^([a-zA-Z]{2}([\-\_][a-zA-Z]{2,4})?)$')
+            meta_regex = re.compile('^METADATA_.+')
+            #does it contay any other property than 'KEY; and is it a valid one?
+            if(len(columnDescription) == 1 or
+                not(
+                    any(prop in columnDescription for prop in valid_properties) or
+                    any(language_regex.match(prop) for prop in columnDescription) or
+                    any(meta_regex.match(prop) for prop in columnDescription)
+                )
+            ):
+                return_data.append({
+                    'path': source['columnDescription'],
+                    'error': 'For \'csv\', \'tsv\' and \'xls\' in \'columnDescription\' property the \'KEY\' and at least one other property like language code, \'<language>\', \'DESCRIPTION\', \'LENGTH\' or metadta is compulsory. \nFor more informations and examples on how todo that please refer to the Applanga CLI Integration Documentation.'
+                })
+                continue
+
+            #are values inide columnDescription only numbers
+            if not all(isinstance(value, int) and value >= 0 for value in columnDescription.values()):
+                return_data.append( {
+                    'path': source['columnDescription'],
+                    'error': 'For \'csv\', \'tsv\' and \'xls\' a;; ptoprties in \'columnDescription\' must have none-negative integer value. \nFor more informations and examples on how todo that please refer to the Applanga CLI Integration Documentation.'
+                })
+                continue
+
         language_files = files.getFiles(source)
 
         files_to_upload.append(language_files['found'])
@@ -256,8 +316,8 @@ def uploadFiles(upload_files, force=False, draft=False, debug=False):
                 if  'disable_plurals' in file_data:
                     send_data['disable_plurals'] = file_data['disable_plurals']
 
-                if 'xliffStatus' in file_data:
-                    send_data['xliffStatus'] = file_data['xliffStatus']
+                if 'importStatus' in file_data:
+                    send_data['importStatus'] = file_data['importStatus']
 
                 if file_data['file_format'] in ['xliff'] and 'skipLockedTranslations' in file_data:
                     send_data['skipLockedTranslations'] = file_data['skipLockedTranslations']
@@ -280,6 +340,14 @@ def uploadFiles(upload_files, force=False, draft=False, debug=False):
                 if file_data['file_format'] in ['xliff'] and 'importSourceLanguage' in file_data:
                     send_data['importSourceLanguage'] = file_data['importSourceLanguage']
 
+                if file_data['file_format'] in ['csv', 'tsv', 'xls']:
+                    if 'includeFirstRow' in file_data:
+                        send_data['includeFirstRow'] = file_data['includeFirstRow']
+                    if 'columnDescription' in file_data:
+                        send_data['columnDescription'] = file_data['columnDescription']
+                    if 'sheetName' in file_data:
+                        send_data['sheetName'] = file_data['sheetName']
+
                 response = uploadFile(send_data, force=force, draft=draft, debug=debug)
                 return_data.append(
                     {
@@ -326,8 +394,8 @@ def uploadFile(file_data, force=False, draft=False, debug=False):
         if file_data['file_format'] in ['xliff'] and 'importSourceLanguage' in file_data:
             request_options['importSourceLanguage'] = file_data['importSourceLanguage']
 
-        if file_data['file_format'] in ['xliff'] and 'xliffStatus' in file_data:
-            request_options['xliffStatus'] = file_data['xliffStatus']
+        if file_data['file_format'] in ['xliff'] and 'importStatus' in file_data:
+            request_options['importStatus'] = file_data['importStatus']
 
         if file_data['file_format'] in ['xliff'] and 'createUnknownCustomStates' in file_data:
             request_options['createUnknownCustomStates'] = file_data['createUnknownCustomStates']
@@ -349,6 +417,15 @@ def uploadFile(file_data, force=False, draft=False, debug=False):
         if 'removeCrChar' in file_data:
             request_options['removeCrChar'] = file_data['removeCrChar']
 
+        if file_data['file_format'] in ['csv', 'tsv', 'xls'] and 'columnDescription' in file_data:
+            spreadsheetOptions = {}
+            spreadsheetOptions['columnDescription'] = file_data['columnDescription']
+            if 'includeFirstRow' in file_data:
+                spreadsheetOptions['includeFirstRow'] = file_data['includeFirstRow']
+            if 'sheetName' in file_data:
+                spreadsheetOptions['sheetName'] = file_data['sheetName']
+            request_options['spreadsheetOptions'] = spreadsheetOptions
+
         request_data = {
             'file-format': file_data['file_format'],
             'language': file_data['language'],

lib/constants.py

@@ -1,4 +1,4 @@
-VERSION_NUMBER = '1.0.104'
+VERSION_NUMBER = '1.0.105'
 APPLANGA_HOST = 'https://api.applanga.com'
 API_BASE_PATH = '/v1/api'
 CONFIG_FILE_NAME = '.applanga.json'
@@ -177,5 +177,23 @@
         'extension': 'xliff',
         'default_file_path': './<language>.xliff',
         'default_tag_name': 'app:language.xliff'
+    },
+    'csv': {
+        'name': 'CSV File',
+        'extension': 'csv',
+        'default_file_path': './<language>.csv',
+        'default_tag_name': 'app:language.csv'
+    },
+    'tsv': {
+        'name': 'TSV File',
+        'extension': 'tsv',
+        'default_file_path': './<language>.tsv',
+        'default_tag_name': 'app:language.tsv'
+    },
+    'xls': {
+        'name': 'Excel',
+        'extension': 'xls',
+        'default_file_path': './<language>.xls',
+        'default_tag_name': 'app:language.xls'
     }
 }

lib/files.py

@@ -148,8 +148,12 @@ def getFiles(source):
             if 'disable_plurals' in source:
                 return_files[file]['disable_plurals'] = source['disable_plurals']
 
+            #'importStatus' overwrites 'xliffStatus' if both are present
             if 'xliffStatus' in source:
-                return_files[file]['xliffStatus'] = source['xliffStatus']
+                return_files[file]['importStatus'] = source['xliffStatus']
+            if 'importStatus' in source:
+                return_files[file]['importStatus'] = source['importStatus']
+
             if 'createUnknownCustomStates' in source:
                 return_files[file]['createUnknownCustomStates'] = source['createUnknownCustomStates']
             if 'importSourceLanguage' in source:
@@ -165,6 +169,18 @@ def getFiles(source):
             if 'importIntoGroup' in source: 
                 return_files[file]['importIntoGroup'] = source['importIntoGroup']
 
+            if 'columnDescription' in source:
+                if '<language>' in source['columnDescription']:
+                    #only replace '<language>' if the same languiages does not yet exist in the 'columnDescription'
+                    if file_language not in source['columnDescription']:
+                        source['columnDescription'][file_language] = source['columnDescription']['<language>']
+                    del source['columnDescription']['<language>']
+                return_files[file]['columnDescription'] = source['columnDescription']
+            if 'sheetName' in source:
+                return_files[file]['sheetName'] = source['sheetName']
+            if 'excludeHeaderRow' in source:
+                return_files[file]['includeFirstRow'] = source['excludeHeaderRow']
+
     return {
                 'skipped': skipped_files,
                 'found': return_files,