feat: Citi Hackathon code submission

[Psingle20]

This PR can be considered as a submission for the FinOS CitiHackathon.
Team members:

• Prachit Ingle Psingle20
• Shabbir Kaderi shabbirflow
• Chaitanya Deshmukh ChaitanyaD48

---

This PR solves issue #745 #788 #796 #797 #765

GITPROXY PLUGINS

We have worked on the following features :

• Sensitive Data Detection ( in files like .json, .xlsx, .csv )
• Check EXIF Metadata from Images ( .jpg, .jpeg, .tiff )
• Detection of AI/ML usage (incl. weights, models etc.)
• Vulnerability Detection using GitLeaks
• Detection of Non-Standard Cryptography Usage

---

Some Modifications Non-Standard Cryptography Usage are required.

"authorisedList": [
    {
      "project": "finos",
      "name": "git-proxy",
      "url": "https://github.com/finos/git-proxy.git"
    },
    {
      "project": "project name",
      "name": "repo name",
      "url": "repo github url",
      "LocalRepoRoot": "specify you local repository path"
    }
  ],

Add the path to your local repository or working directory in the localRepoRoot in the authorisedList to give git-proxy access to your files.

Sensitive Data Detection ( in files like .json, .xlsx, .csv )

Features:
This solves issue #745

• https://github.com/Psingle20/git-proxy/blob/CitiHackathon/src/proxy/processors/push-action/checkSensitiveData.js - This file contains the detection logic
• For this to work, it is required to configure proxy.config.json with the file types for which sensitive data detection is required, for ex:

    "diff": {
      "block": {
        "literals": [],
        "patterns": [],
        "providers": {},
        "proxyFileTypes": [".csv", ".xlsx", ".log", ".json"]
      }
    },

• https://github.com/Psingle20/git-proxy/blob/CitiHackathon/test/CheckSensitive.test.js - Relevant tests are mentioned in this file

---

Check EXIF Metadata from Images ( .jpg, .jpeg, .tiff )

Features:
This solves issue #796

• https://github.com/Psingle20/git-proxy/blob/CitiHackathon/src/proxy/processors/push-action/checkExifJpeg.js - This file contains the logic for EXIF Metadata retrieval and parsing.
• The user can configure proxy.config.json with the file types for which EXIF Metadata needs to be detected.

    "diff": {
      "block": {
        "literals": [],
        "patterns": [],
        "providers": {},
        "proxyFileTypes": [".jpg", ".jpeg", ".tiff"]
      }
    },

• This will block push event if sensitive metadata such as Geolocation information or Camera Make or Model ( or other sensitive information ) is detected.
• Relevant tests are described in https://github.com/Psingle20/git-proxy/blob/CitiHackathon/test/CheckExif.test.js

---

Detection of AI/ML usage (incl. weights, models etc.)

Features:
This solves issue #788

• https://github.com/Psingle20/git-proxy/blob/CitiHackathon/src/proxy/processors/push-action/checkForAiMlUsage.js - This file contains the logic for AI/ML usage detection based on file extensions of model weights, usage of popular ML libraries, common AI functions usage, configuration keys, etc.
• The user can configure proxy.config.json with the parameters for which detection needs to be carried out.

    "aiMlUsage": {
          "enabled": true,
          "blockPatterns": ["modelWeights", "largeDatasets", "aiLibraries", "configKeys", "aiFunctions"]
    }

• https://github.com/Psingle20/git-proxy/blob/CitiHackathon/test/checkAiMlUsage.test.js - Relevant tests are mentioned in this file

---

Vulnerability Detection using GitLeaks

Features:
This solves issue #797

• We have used GitLeaks to detect vulnerabilities in the codebase.
• Checks have been added in gitleaks.toml file
• The main logic of the implementation lies in https://github.com/Psingle20/git-proxy/blob/CitiHackathon/src/proxy/processors/push-action/checkForSecrets.js file
• The user can configure proxy.config.json to enable / disable the plugin.

    "checkForSecrets": {
      "enabled": false
    },

• A detailed report will be generated gitleaks_reports.json
• Some modifications / minor changes might be required for this to be merged.

---

Detection of Non-Standard Cryptography Usage

This solves issue #765

Features:

• The logic for non - Standard cryptography detection lies in the file https://github.com/Psingle20/git-proxy/blob/CitiHackathon/src/proxy/processors/push-action/checkCryptoImplementation.js
• Relevant tests for core functionality have been described in https://github.com/Psingle20/git-proxy/blob/CitiHackathon/test/checkCryptoImplementation.test.js
• Some modifications / minor changes might be required for this to be merged.

[netlify]

✅ Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name	Link
🔨 Latest commit	cca6713
🔍 Latest deploy log	https://app.netlify.com/sites/endearing-brigadeiros-63f9d0/deploys/67480028921c3c0008ee0d93

[Psingle20]

@JamieSlome @coopernetes we were working on some refactor on #798 PR and due to some merge conflicts we had to rollback and the PR got closed this PR contains all the commits we have done till 14-11-2024 please consider this as our submission . We will demonstrate it during our presentation.

you can check the commit history for the dates of hackathon period.

[Psingle20]

@rgmz I have updated the gitleaks rules can you review it and suggest any other changes if needed?

[jescalada]

@Psingle20 Thanks for the contribution! 🚀

It might be helpful to split this PR into smaller chunks for each of the issues solved. That will make it easier for us to review 🙂

I've checked out the #745 implementation only.

[jescalada]

It seems this implementation isn't quite what was mentioned in #745. The goal of #745 is to allow blocking certain filenames, such as for example this-is-a-secret.js. We want to be able to use patterns to match these, like we can already do with commit messages and diffs.

The goal is to extend the commitConfig entry in proxy.config.json to include a filename entry:

"commitConfig": {
    "author": {
      "email": {
        "local": {
          "block": ""
        },
        "domain": {
          "allow": ".*"
        }
      }
    },
    "message": {
      "block": {
        "literals": [],
        "patterns": []
      }
    },
    "diff": {
      "block": {
        "literals": [],
        "patterns": [],
        "providers": {}
      }
    },
    "filename": {
      "block": {
        "literals": [],
        "patterns": []
      }
    }
  },

And then implement the logic for this in a push-action file, such as checkFilenames.js.

[rgmz]

The Gitleaks portion of this PR might be superseded by #1010.

[Psingle20]

Hi @jescalada,

This PR was initially created to gather all the features developed during the hackathon in one place. While we could raise a new PR, I believe there have been several new features added since the hackathon, so we’ll need to adjust accordingly.

Also, as @coopernetes suggested, instead of adding it to chain , we should consider refactoring these into plugins. That might require some restructuring to align with the new direction.