alonbl patch series (#1017)

* Add linesep configuration option Platform independent line ends are important to avoid conflicts. Resolves: #1010 Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com> * Perform atomic write to files Do not overwrite file as we may corrupt the file if something fails, instead write into temporary location and move the result into the final destination. Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com> * Synchronize README command-line usage Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com> * Support files as positional argument Tools such as pre-commit and other tools expects the tools to accept the input files as positional argument and not as options. Let's support the two variants. Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com> * Support shallow git clone without tags pre-commit uses shallow clone without tags, as this is not used to publish the artifact, let's assume version '0' in this case. Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com> * Add pre-commit hooks support Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com> --------- Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
jeremiah-c-leary · Nov 5, 2023 · aeeeed6 · aeeeed6
1 parent 941a661
commit aeeeed6
Show file tree

Hide file tree

Showing 17 changed files with 190 additions and 36 deletions.
diff --git a/.pre-commit-hooks.yaml b/.pre-commit-hooks.yaml
@@ -0,0 +1,8 @@
+- id: vsg
+  name: fix VHDL style
+  description: VHDL Style Guide
+  entry: vsg --fix --jobs=1
+  language: python
+  types:
+    - file
+  files: \.(vhd|vhdl)$
diff --git a/README.rst b/README.rst
@@ -158,12 +158,16 @@ The command line tool can be invoked with:
    $ vsg
    usage: VHDL Style Guide (VSG) [-h] [-f FILENAME [FILENAME ...]] [-lr LOCAL_RULES] [-c CONFIGURATION [CONFIGURATION ...]] [--fix]
                                  [-fp FIX_PHASE] [-j JUNIT] [-js JSON] [-of {vsg,syntastic,summary}] [-b] [-oc OUTPUT_CONFIGURATION]
-                                 [-rc RULE_CONFIGURATION] [--style {indent_only,jcl}] [-v] [-ap] [--fix_only FIX_ONLY] [-p JOBS]
-                                 [--debug]
-   
+                                 [-rc RULE_CONFIGURATION] [--style {indent_only,jcl}] [-v] [-ap] [--fix_only FIX_ONLY] [--stdin]
+                                 [--quality_report QUALITY_REPORT] [-p JOBS] [--debug]
+                                 [FILENAME ...]
+
    Analyzes VHDL files for style guide violations. Reference documentation is located at: http://vhdl-style-guide.readthedocs.io/en/latest/index.html
-   
-   optional arguments:
+
+   positional arguments:
+     FILENAME              File to analyze
+
+   options:
      -h, --help            show this help message and exit
      -f FILENAME [FILENAME ...], --filename FILENAME [FILENAME ...]
                            File to analyze
@@ -190,13 +194,29 @@ The command line tool can be invoked with:
      -v, --version         Displays version information
      -ap, --all_phases     Do not stop when a violation is detected.
      --fix_only FIX_ONLY   Restrict fixing via JSON file.
+     --stdin               Read VHDL input from stdin, disables all other file selections, disables multiprocessing
+     --quality_report QUALITY_REPORT
+                           Create code quality report for GitLab
      -p JOBS, --jobs JOBS  number of parallel jobs to use, default is the number of cpu cores
      --debug               Displays verbose debug information
 
 Here is an example output running against a test file:
 
 .. image:: https://github.com/jeremiah-c-leary/vhdl-style-guide/blob/master/docs/img/fixing_single_file.gif
 
+pre-commit Integration
+----------------------
+
+Here is an example of ``.pre-commit-config.yaml`` file:
+
+.. code-block:: yaml
+
+  repos:
+    - repo: https://github.com/jeremiah-c-leary/vhdl-style-guide
+      rev: v3.18.0
+      hooks:
+        - id: vsg
+
 Documentation
 -------------
 

diff --git a/docs/configuring_overview.rst b/docs/configuring_overview.rst
@@ -97,6 +97,17 @@ The file_rules option allows for configuration of individual rules per file.
 Any file listed under this option will have the configuration applied if it is being analyzed.
 .. The file_rules is exactly the same as file_list except that it will not add the file to the scan list.
 
+linesep
+-------
+
+The linesep is an optional settings for line separator.
+Default is platform specific.
+Logical values may be "\n" or "\r\n".
+
+.. code-block:: yaml
+
+   linesep: "\n"
+
 local_rules
 -----------
 

diff --git a/docs/index.rst b/docs/index.rst
@@ -20,7 +20,7 @@ Welcome to vhdl-style-guide's documentation!
    code_tags
    continuous_integration_servers
    editor_integration/editor_integration
-   tool_integration
+   tool_integration/tool_integration
    pragmas
    localizing
    phases

diff --git a/docs/tool_integration.rst → docs/tool_integration/generic.rst b/docs/tool_integration.rst → docs/tool_integration/generic.rst
@@ -1,5 +1,5 @@
-Tool Integration
-----------------
+Generic Tool Integration
+------------------------
 
 VSG supports integration with other tools via several command line options.
 

diff --git a/docs/tool_integration/pre-commit.rst b/docs/tool_integration/pre-commit.rst
@@ -0,0 +1,25 @@
+pre-commit Integration
+----------------------
+
+VSG supports integration with pre-commit using the following ``.pre-commit-config.yaml`` file:
+
+.. code-block:: yaml
+
+  repos:
+    - repo: https://github.com/jeremiah-c-leary/vhdl-style-guide
+      rev: v3.18.0
+      hooks:
+        - id: vsg
+
+You may customize VSG by using the ``arg`` node, for example:
+
+.. code-block:: yaml
+
+  repos:
+    - repo: https://github.com/alonbl/vhdl-style-guide
+      rev: v3.18.0
+      hooks:
+        - id: vsg
+          args:
+            - --configuration=.vsg.yaml
+            - --output_format=syntastic
diff --git a/docs/tool_integration/tool_integration.rst b/docs/tool_integration/tool_integration.rst
@@ -0,0 +1,8 @@
+Tool Integration
+------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   generic.rst
+   pre-commit.rst
diff --git a/docs/usage.rst b/docs/usage.rst
@@ -14,14 +14,18 @@ The command line tool can be invoked with:
                                  [-of {vsg,syntastic,summary}] [-b]
                                  [-oc OUTPUT_CONFIGURATION]
                                  [-rc RULE_CONFIGURATION]
-                                 [--style {jcl,indent_only}] [-v] [-ap]
+                                 [--style {indent_only,jcl}] [-v] [-ap]
                                  [--fix_only FIX_ONLY] [--stdin]
                                  [--quality_report QUALITY_REPORT] [-p JOBS]
                                  [--debug]
-   
+                                 [FILENAME ...]
+
    Analyzes VHDL files for style guide violations. Reference documentation is
    located at: http://vhdl-style-guide.readthedocs.io/en/latest/index.html
-   
+
+   positional arguments:
+     FILENAME              File to analyze
+
    options:
      -h, --help            show this help message and exit
      -f FILENAME [FILENAME ...], --filename FILENAME [FILENAME ...]
@@ -45,7 +49,7 @@ The command line tool can be invoked with:
                            Write configuration to file name.
      -rc RULE_CONFIGURATION, --rule_configuration RULE_CONFIGURATION
                            Display configuration of a rule
-     --style {jcl,indent_only}
+     --style {indent_only,jcl}
                            Use predefined style
      -v, --version         Displays version information
      -ap, --all_phases     Do not stop when a violation is detected.

diff --git a/vsg/apply_rules.py b/vsg/apply_rules.py
@@ -125,7 +125,7 @@ def apply_rules(commandLineArguments, oConfig, tIndexFileName):
         oRules.fix(
             commandLineArguments.fix_phase, commandLineArguments.skip_phase, fix_only
         )
-        write_vhdl_file(oVhdlFile)
+        write_vhdl_file(oVhdlFile, oConfig.dConfig)
 
     oRules.clear_violations()
     oRules.check_rules(
@@ -149,13 +149,20 @@ def apply_rules(commandLineArguments, oConfig, tIndexFileName):
     return fExitStatus, testCase, dJsonEntry, sOutputStd, sOutputErr, bKeepProcessingFiles
 
 
-def write_vhdl_file(oVhdlFile):
+def write_vhdl_file(oVhdlFile, dConfig):
+    tmpfile = f"{oVhdlFile.filename}.tmp"
     try:
-        with open(oVhdlFile.filename, 'w', encoding='utf-8') as oFile:
+        with open(tmpfile, 'w', encoding='utf-8', newline=dConfig.get("linesep")) as oFile:
             for sLine in oVhdlFile.get_lines()[1:]:
                 oFile.write(sLine + '\n')
+        os.replace(tmpfile, oVhdlFile.filename)
     except PermissionError as err:
         print (err, "Could not write fixes back to file.")
+    finally:
+        try:
+            os.remove(tmpfile)
+        except FileNotFoundError:
+            pass
 
 
 def create_junit_testcase(sVhdlFileName, oException):

diff --git a/vsg/cmd_line_args.py b/vsg/cmd_line_args.py
@@ -48,7 +48,7 @@ def parse_command_line_arguments():
                    Reference documentation is located at:
                    http://vhdl-style-guide.readthedocs.io/en/latest/index.html''')
 
-    parser.add_argument('-f', '--filename', type=__is_valid_file, nargs='+', help='File to analyze')
+    parser.add_argument('-f', '--filename', type=__is_valid_file, nargs='+', default=[], help='File to analyze')
     parser.add_argument('-lr', '--local_rules', help='Path to local rules')
     parser.add_argument('-c', '--configuration', type=__is_valid_file, nargs='+', help='JSON or YAML configuration file(s)')
     parser.add_argument('--fix', default=False, action='store_true', help='Fix issues found')
@@ -82,6 +82,8 @@ def parse_command_line_arguments():
         help="number of parallel jobs to use, default is the number of cpu cores",
     )
     parser.add_argument('--debug', default=False, action='store_true', help='Displays verbose debug information')
+    parser.add_argument('filename_args', metavar='FILENAME', type=__is_valid_file, nargs='*', default=[],
+                        help='File to analyze')
 
     args_ = parser.parse_args()
 
@@ -117,7 +119,7 @@ def get_predefined_styles():
             with open(os.path.join(sStylePath, sStyle)) as yaml_file:
                 tempConfiguration = yaml.safe_load(yaml_file)
             lReturn.append(tempConfiguration['name'])
-    return lReturn
+    return sorted(lReturn)
 
 
 def validate_backup_argument(args_):
@@ -147,12 +149,12 @@ def add_quality_report_argument(parser):
 
 
 def fix_filename_argument(args_):
-    if args_.filename is None:
-        return None
     lUpdate = []
     for lFilenames in args_.filename:
         lUpdate.extend(lFilenames)
-    args_.filename = lUpdate
+    for lFilenames in args_.filename_args:
+        lUpdate.extend(lFilenames)
+    args_.filename = lUpdate or None
 
 
 def fix_configuration_argument(args_):

diff --git a/vsg/tests/cmd_line_args/test_cmd_line_args.py b/vsg/tests/cmd_line_args/test_cmd_line_args.py
@@ -48,6 +48,32 @@ def test_filename_w_two_files(self):
 
         self.assertEqual(lActual, lExpected)
 
+    def test_filename_w_two_files_positional(self):
+        sys.argv = ['vsg']
+        sys.argv.extend([sAFile, sBFile])
+
+        lExpected = [sAFile, sBFile]
+        lExpected.sort()
+
+        oActual = cmd_line_args.parse_command_line_arguments()
+        lActual = oActual.filename
+        lActual.sort()
+
+        self.assertEqual(lActual, lExpected)
+
+    def test_filename_w_two_files_positional_merge(self):
+        sys.argv = ['vsg']
+        sys.argv.extend(['-f', sAFile, sBFile])
+
+        lExpected = [sAFile, sBFile]
+        lExpected.sort()
+
+        oActual = cmd_line_args.parse_command_line_arguments()
+        lActual = oActual.filename
+        lActual.sort()
+
+        self.assertEqual(lActual, lExpected)
+
     def test_valid_glob(self):
         sys.argv = ['vsg']
         sys.argv.extend(['-f', sGlobFile])

diff --git a/vsg/tests/vhdlFile/format_ansi/classification_results.txt b/vsg/tests/vhdlFile/format_ansi/classification_results.txt
@@ -0,0 +1,5 @@
+--------------------------------------------------------------------------------
+0 |
+--------------------------------------------------------------------------------
+1 | -- ANSI "" comment
+<class 'vsg.parser.comment'>
diff --git a/vsg/tests/vhdlFile/format_ansi/classification_test_input.vhd b/vsg/tests/vhdlFile/format_ansi/classification_test_input.vhd
@@ -0,0 +1 @@
+-- ANSI "���" comment
diff --git a/vsg/tests/vhdlFile/test_format_ansi.py b/vsg/tests/vhdlFile/test_format_ansi.py
@@ -0,0 +1,28 @@
+import os
+
+import unittest
+
+from vsg.tests import utils
+from vsg import vhdlFile
+
+sLrmUnit = utils.extract_lrm_unit_name(__name__)
+
+lFile, eError =vhdlFile.utils.read_vhdlfile(os.path.join(os.path.dirname(__file__), sLrmUnit,'classification_test_input.vhd'))
+oFile = vhdlFile.vhdlFile(lFile)
+
+
+class test_token(unittest.TestCase):
+
+    def test_classification(self):
+        sTestDir = os.path.join(os.path.dirname(__file__), sLrmUnit)
+
+        lExpected = []
+        utils.read_file(os.path.join(sTestDir, 'classification_results.txt'), lExpected, False)
+
+        lActual = []
+
+        for oObject in utils.extract_objects(oFile, True):
+            lActual.append(str(oObject))
+
+        self.assertEqual(lExpected, lActual)
+
diff --git a/vsg/tests/vsg/test_vsg.py b/vsg/tests/vsg/test_vsg.py
@@ -520,3 +520,13 @@ def test_configuration_with_file_rules_and_no_file_list_entity1(self):
             iExitStatus = e.returncode
 
         self.assertEqual(lActual, lExpected)
+
+    def test_file_as_stdin(self):
+
+        with open('vsg/tests/vsg/entity1.vhd') as file1:
+            lExpected = []
+            lExpected.append('')
+
+            lActual = subprocess.check_output(['bin/vsg','--configuration','vsg/tests/vsg/config_3.yaml','--output_format','syntastic','--stdin'], stdin=file1)
+            lActual = str(lActual.decode('utf-8')).split('\n')
+            self.assertEqual(lActual, lExpected)
diff --git a/vsg/version.py b/vsg/version.py
@@ -33,13 +33,14 @@ def get_version_info():
         sReturnPath = os.getcwd()
         sPath = os.path.dirname(__file__)
         os.chdir(sPath)
-        lActual = subprocess.check_output(['git', 'describe', '--tags'])
-        lActual = str(lActual.decode('utf-8')).split('\n')
-        lVersion = lActual[0].split('-')
+        lVersion = "0"  # version if no tags (shallow checkout)
         try:
+            lActual = subprocess.check_output(['git', 'describe', '--tags'])
+            lActual = str(lActual.decode('utf-8')).split('\n')
+            lVersion = lActual[0].split('-')
             sVersion = str(lVersion[0]) + '.dev' + str(lVersion[1])
             sShaNum = str(lVersion[-1][1:])
-        except IndexError:
+        except (IndexError, subprocess.CalledProcessError):
             sVersion = str(lVersion[0])
             lActual = subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD'])
             lActual = str(lActual.decode('utf-8')).split('\n')

diff --git a/vsg/vhdlFile/utils.py b/vsg/vhdlFile/utils.py
@@ -757,23 +757,21 @@ def is_whitespace(oObject):
 
 
 def read_vhdlfile(sFileName):
-    if sFileName == 'stdin':
+
+    def _read(oFile):
         lLines = []
-        for line in sys.stdin:
-            lLines.append(line[:-1])
-        return lLines, None
+        for sLine in oFile:
+            lLines.append(sLine.rstrip('\r\n'))
+        return lLines
+
+    if sFileName == 'stdin':
+        return _read(sys.stdin), None
     try:
-        lLines = []
         with open(sFileName, encoding='utf-8') as oFile:
-            for sLine in oFile:
-                lLines.append(sLine)
-        return lLines, None
+            return _read(oFile), None
     except UnicodeDecodeError:
-        lLines = []
         with open(sFileName, encoding="ISO-8859-1") as oFile:
-            for sLine in oFile:
-                lLines.append(sLine)
-        return lLines, None
+            return _read(oFile), None
     except OSError as e:
         return [], e