Improvements to aid in community contributions and adoption

[achrinza]

This is a very loaded issue, and probably should be broken down into separate issues. For now, it's one issue so as to not look "spammy".

---

Table of Contents

• Add CONTRIBUTING.md
• Add SECURITY.md
• Add external core contributors
• Adopt Contributor Covenant 2.0 Code of Conduct
• Adopt Developer Certificate of Origin
• Adopt All Contributors specification
• Adopt Cloud Native Module LTS policy
• Use ESlint and Prettier
• Abstract away from Google Firebase
• Publish to NPM
• Utilize a monorepo structure
• Utilize a proper dependency injection system
• Utilize PR testing tooling
• Utilize static vulnerability scanning platforms
• Utilize dynamic code analysis platforms
• Utilize license scanning tools
• Switch to a more open license (Apache 2.0 or MIT)?
• Add OpenAPI 3.0 Spec docs
• Utilize proper logging tools instead of console.*()
• Code style: do not allow anonymous functions

---

Currently, there are many areas of improvement to encourage contributions to the OpenTrace projects. This may cause friction in community contributions and adoption.

Some potential actionable things we can do:

• Add CONTRIBUTING.md

  Explain...

  1. How to create a new PR
     a. Should we squash our commits?
     b. Does the project adopt a certain commit message style (e.g. Conventional Commits)
     c. Which code editor is recommended, and which extensions? (e.g vscode/vscodium with ESlint and Prettier extensions)
  2. Code style guide
  3. Does the project adopt a CLA or DCO?

• Add SECURITY.md

  Having a well-defined responsible disclosure workflow is important. The Node.js seucirty WG Responsible Disclosure Template can be the baseline for our SECURITY.md.

• Add external core contributors

  Correct me if I'm wrong, but this project is managed by people in GovTech or people with close connections to GovTech. It may be a good idea to bring in other people from outside as core contributors to increase support towards the community.

• Adopt Contributor Covenant 2.0 Code of Conduct

  The "CC 2.0 CoC" helps to formalise the contributors' code of conduct and provide an avenue of reporting violations of the CoC.

• Adopt Developer Certificate of Origin (CLA vs DCO)

  A DCO will help reduce uncertainties on the legality of contributions by the community. It's also more "open" than a CLA.

  DCO Signed-off-by can be enforced with the GitHub Probot DCO app.

• Adopt All Contributors specification

  This will encourage community contributions as it allow them to be given visible attribution. It's adopted by several other open source projects.

• Adopt Cloud Native Module LTS policy.

  This is useful to ensuring support of Node.js LTS and Current versions. This shouldn't be much of an issue as the project uses Typescript.

• Use ESlint and Prettier

  To ensure code consistency, ESlint and Prettier should be adopted. They're arguably the 2 most common tools used for this purpose.

  Also, tslint has been deprecated.

• Abstract away from Google Firebase

  The project is currently tightly-coupled with Firebase. It may not be feasible for every organisation to adopt and it puts reliance onto one vendor.

  We can abstract the core components of OpenTrace and then allow vendor-specific wrappers to adopt this common core.

  There are other FaaS services that other people may choose to adopt:

  • Apache OpenWhisk
  • Severless Framework
  • Kubeless
  • Knative

  Furthermore, other users may want to use a non-FaaS model:

  • Cloud Foundry (PaaS)
  • K8s/Docker (Containers)

  In Addition, users may want to use a different storage service:

  • AWS S3
  • OpenStack Swift
  • Azure Blob Storage

  Note: We don't need to support these tools directly, but we need to enable users to be able to adapt the application to support it. We can achieve this by creating well-defined APIs for wrappers to utilize.

• Publish to NPM

  Linking back to the previous point above, we should publish the common core of OpenTrace to NPM to allow other projects to more easily integrate OpenTrace.

• Utilize a monorepo structure

  Linking back to the previous 2 points above, we should utilize a monorepo structure to manage inter-packages dependencies. This ease the management of separate wrappers for different vendors and allowing them to depend on the common core.

• Utilize a proper dependency injection system

  Currently, there's no DI. This means that the extension points aren't as clear as they could be. One example is processUploadedData.ts. It requires direct modification of the file which may be a fragile API as the project continues to evolve.

• Utilize PR testing tooling

  Linking back to the point about having a common core for OpenTrace, we should utilize the free tooling provided for open source projects. One case is PR testing with GitHub Actions (which is based off Azure Pipelines). It's free for open source projects and can help with:

  • Preventing regressions
  • Reducing effort to contribute

• Utilize dependecy updater tools

  Node.js' design means that each project tends to have many dependencies. This makes it difficult to keep them up-to-date consistently. Free tools such as WhiteSource Renovate can help ensure that dependencies and package locks are up-to-date.

• Utilize static vulnerability scanning platforms

  Snyk is a common tool for this. This will ensure that the project does not have any known vulnerabilities

• Utilize dynamic code analysis platforms

  Dynamic code analysis platforms can mitigate potentially dangerous code from being contributed into the project. Semmle LGTM and DeepScan are the commonly-used tool.

• Utilize license scanning tools

  As Node.js projects utilize many direct and indirect dependencies, it is difficult to ensure compliance and compatibility of the license. FOSSA is a common SAAS tool to check for license issues.

• Switch to a more open license (Apache 2.0 or MIT)?

  GPL 3.0 can make it difficult for certain users (or other projects) to adopt the project. This can be due to restricted IPs or policies.

  • While Apache 2.0-licensed code can be used in a GPL v3.0-licensed projects, the other way is not possible.

  • Many projects (notwithstanding Linux and alike using GPL v2.0) use Apache 2.0 or MIT

  • Apache 2.0 (or even better, MIT) is more compatible with other licenses.

  • Case in point, Twitter Bootstrap had to re-license to MIT to support Drupal's GPL license.

  • Another case is GPL license potentially problematic for App Store distribution opentrace-ios#2

• Add OpenAPI 3.0 Spec docs

  There's no OAS3 Spec docs, which prevents users from utilizing code generation tools that leverage the OAS3 Spec docs.

• Utilize proper logging tools instead of console.*()

  Tools such as Winston help ensure that logging is managed properly. This can prevent code bloat, allow re-routing of logs to another consumer, and properly tag and filter by log levels.

• Code style: do not allow anonymous functions

  Anonymous functions would obfuscate the stack trace should an error occur.
  An example of anonymous functions being used in this project

This isn't complete list nor a must-have by any means, but it's a good discussion starting point.

[qtangs]

Thanks @achrinza for the detailed suggestions. We appreciate all the points. Our team is in the process of working out these areas along with many other priorities. We will provide update when possible. Stay tuned.

[achrinza]

Thanks @qtangs; I do appreciate the detailed work that the TraceTogether team has done, especially those related to overcoming the issues with Bluetooth.

In the mean time, are community contributions being accepted? And are community maintainers welcomed?

Some PRs are already opened across the different repos. It'd be great if we can quickly merge them and not let them go stale so as to leverage the initial wave of community interest around the project.

I do believe that OpenTrace can be incredibly useful, and we should allow the community to contribute and adopt with little friction.

I look forward to helping improve OpenTrace along with the rest of the open source community.

[siilats]

Hi, is there a plan to provide a reference implementation of processUploadedData.ts as well? Its a bit hard to follow how the keys are matched without it.
const processUploadedData = async (object: ObjectMetadata) => {
throw new functions.https.HttpsError('unimplemented', 'Not implemented yet');
};

[achrinza]

Hi @qtangs; Some PRs have been opened up in this repo to resolve some of the issues raised in this issue.