feat: set component versions to 0.1.0-alpha (#123)

Because

- We need a way to introduce breaking changes in a sensible way.
- Component page will need the version and release stage of components.

This commit

- Adds a `version` field to the component definition. Initial version is
`0.1.0-alpha` as we don't have a way to migrate components at the moment
and
  thus we can't guarantee that we won't introduce more breaking changes.
- Moves each component package to a `v0` directory so more than one
major
  version can coexist.
- Introduces the `source_url` field in the component definition. This
field is
  required by the component page and will be returned in the connector /
  operator definition list.

## Notes

Adding the `source_url` field in the definition isn't great as it
references a
particular branch (in the `blob/main` bit) and because writing it by
hand
shouldn't be required. It would be great if we could just build it from
the
component ID but several connectors (e.g. `gcp` or `instill-model`)
don't match
the package and the ID. Also, there's no clean way to build this value
and
inject it into the `Load[Connector|Operator]Definitions` function.

To make the definitions file easier to build and less error-prone, we
should
introduce a linter and perhaps a boilerplate builder in the future.

### 🤖 Script to migrate to v0

```sh
#!/usr/bin/env bash

set -eo pipefail

# This script modifies the directory of each component and sets their version
# to 0.1.0-alpha. It automatically updates the references to that package.
#
# Usage: sh newversion.sh

for p in $(fd definitions.json | cut -d '/' -f1-2 | sort | uniq); do

  # Move component into v0 subdirectory
  mkdir v0
  mv $p/* v0
  mv v0 $p/v0

  # Update references
  escaped=$(echo $p | sed 's/\//\\\//g')
  grep -rl github.com/instill-ai/connector/$p | sort | uniq | xargs -I '{}' sed -i '' "s/\(github.com\/instill-ai\/connector\/$escaped\)/\1\/v0/g" {}

  # Add version and source URL to definition
  v=0.1.0-alpha
  url=github.com/instill-ai/connector/blob/main/$p/v0

  jq ".[0] += {version: \"$v\", source_url: \"$url\"}" $p/v0/config/definitions.json > def.json
  mv def.json $p/v0/config/definitions.json

done;
```

go.mod

@@ -16,8 +16,8 @@ require (
 	github.com/gocolly/colly/v2 v2.1.0
 	github.com/gofrs/uuid v4.4.0+incompatible
 	github.com/h2non/filetype v1.1.3
-	github.com/instill-ai/component v0.10.0-beta
-	github.com/instill-ai/protogen-go v0.3.3-alpha.0.20240105094938-3a71d8f7a812
+	github.com/instill-ai/component v0.10.0-beta.0.20240206082535-b067f9cbaa98
+	github.com/instill-ai/protogen-go v0.3.3-alpha.0.20240206062817-a862071d8ece
 	github.com/instill-ai/x v0.3.0-alpha.0.20231219052200-6230a89e386c
 	github.com/redis/go-redis/v9 v9.3.0
 	go.uber.org/zap v1.26.0

go.sum

@@ -145,10 +145,10 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.2 h1:I/pwhnUln5wbMnTyRbzswA0/JxpK
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.2/go.mod h1:lsuH8kb4GlMdSlI4alNIBBSAt5CHJtg3i+0WuN9J5YM=
 github.com/h2non/filetype v1.1.3 h1:FKkx9QbD7HR/zjK1Ia5XiBsq9zdLi5Kf3zGyFTAFkGg=
 github.com/h2non/filetype v1.1.3/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
-github.com/instill-ai/component v0.10.0-beta h1:cmYZnIqihVMF96lY2alAZEVjmDwBzw54fq8b2j0a4O4=
-github.com/instill-ai/component v0.10.0-beta/go.mod h1:Vx/D38SiH2gukPSI6FCr6wnhofPSapL0V3uxDHbUyi8=
-github.com/instill-ai/protogen-go v0.3.3-alpha.0.20240105094938-3a71d8f7a812 h1:n1EQFT0NeXkmiCD/YtoLG+b/OPo2tNg7MVK0dHNFlvk=
-github.com/instill-ai/protogen-go v0.3.3-alpha.0.20240105094938-3a71d8f7a812/go.mod h1:jhEL0SauySMoPLVvx105DWyThju9sYTbsXIySVCArmM=
+github.com/instill-ai/component v0.10.0-beta.0.20240206082535-b067f9cbaa98 h1:3B9NVpATwjMuOXA3YXDcofZXZscaezxZir4D1NbGQFw=
+github.com/instill-ai/component v0.10.0-beta.0.20240206082535-b067f9cbaa98/go.mod h1:THyROt2dqqge2lDc+PVzm/+LueG4dBtxpLzSjl+T83A=
+github.com/instill-ai/protogen-go v0.3.3-alpha.0.20240206062817-a862071d8ece h1:Zs8zy+ZgLe8bWRh9E48JaPTB+iS2E7f27OiD0Jjjld8=
+github.com/instill-ai/protogen-go v0.3.3-alpha.0.20240206062817-a862071d8ece/go.mod h1:jhEL0SauySMoPLVvx105DWyThju9sYTbsXIySVCArmM=
 github.com/instill-ai/x v0.3.0-alpha.0.20231219052200-6230a89e386c h1:a2RVkpIV2QcrGnSHAou+t/L+vBsaIfFvk5inVg5Uh4s=
 github.com/instill-ai/x v0.3.0-alpha.0.20231219052200-6230a89e386c/go.mod h1:L6jmDPrUou6XskaLXZuK/gDeitdoPa9yE8ONKt1ZwCw=
 github.com/jawher/mow.cli v1.1.0/go.mod h1:aNaQlc7ozF3vw6IJ2dHjp2ZFiA4ozMIYY6PyuRJwlUg=

pkg/airbyte/config/definitions.json → pkg/airbyte/v0/config/definitions.json

@@ -729,7 +729,7 @@
               },
               "transformation_priority": {
                 "default": "interactive",
-                "description": "Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type <a href=\"https://cloud.google.com/bigquery/docs/running-queries#queries\">here</a>. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don\u2019t count towards your concurrent rate limit. Read more about batch queries <a href=\"https://cloud.google.com/bigquery/docs/running-queries#batch\">here</a>. The default \"interactive\" value is used if not set explicitly.",
+                "description": "Interactive run type means that the query is executed as soon as possible, and these queries count towards concurrent rate limit and daily limit. Read more about interactive run type <a href=\"https://cloud.google.com/bigquery/docs/running-queries#queries\">here</a>. Batch queries are queued and started as soon as idle resources are available in the BigQuery shared resource pool, which usually occurs within a few minutes. Batch queries don’t count towards your concurrent rate limit. Read more about batch queries <a href=\"https://cloud.google.com/bigquery/docs/running-queries#batch\">here</a>. The default \"interactive\" value is used if not set explicitly.",
                 "enum": [
                   "interactive",
                   "batch"
@@ -3484,7 +3484,7 @@
                     "title": "HiveCatalog: Use Apache Hive MetaStore"
                   },
                   {
-                    "description": "A Hadoop catalog doesn\u2019t need to connect to a Hive MetaStore, but can only be used with HDFS or similar file systems that support atomic rename.",
+                    "description": "A Hadoop catalog doesn’t need to connect to a Hive MetaStore, but can only be used with HDFS or similar file systems that support atomic rename.",
                     "properties": {
                       "catalog_type": {
                         "default": "Hadoop",
@@ -4203,7 +4203,7 @@
               "endpoint": {
                 "description": "AWS Kinesis endpoint.",
                 "examples": [
-                  "kinesis.us\u2011west\u20111.amazonaws.com"
+                  "kinesis.us‑west‑1.amazonaws.com"
                 ],
                 "order": 0,
                 "title": "Endpoint",
@@ -4219,7 +4219,7 @@
               "region": {
                 "description": "AWS region. Your account determines the Regions that are available to you.",
                 "examples": [
-                  "us\u2011west\u20111"
+                  "us‑west‑1"
                 ],
                 "order": 1,
                 "title": "Region",
@@ -11926,6 +11926,8 @@
       "airbyte-destination-xata": "airbyte/destination-xata:0.1.1",
       "airbyte-destination-yugabytedb": "airbyte/destination-yugabytedb:0.1.1",
       "airbyte-devmate-cloud": "ghcr.io/devmate-cloud/streamr-airbyte-connectors:0.0.1"
-    }
+    },
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/airbyte/v0"
   }
 ]

pkg/archetypeai/config/definitions.json → pkg/archetypeai/v0/config/definitions.json

@@ -36,6 +36,8 @@
     "type": "CONNECTOR_TYPE_AI",
     "uid": "e414a1f8-5fdf-4292-b050-9f9176254a4b",
     "vendor": "Archetype AI",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/archetypeai/v0"
   }
 ]

pkg/bigquery/config/definitions.json → pkg/bigquery/v0/config/definitions.json

@@ -56,6 +56,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "e2ffe076-ab2c-4e5e-9587-a613a6b1c146",
     "vendor": "Google",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/bigquery/v0"
   }
 ]

pkg/googlecloudstorage/config/definitions.json → pkg/googlecloudstorage/v0/config/definitions.json

@@ -43,6 +43,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "205cbeff-6f45-4abe-b0a8-cec1a310137f",
     "vendor": "Google",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/googlecloudstorage/v0"
   }
 ]

pkg/googlesearch/config/definitions.json → pkg/googlesearch/v0/config/definitions.json

@@ -42,6 +42,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "2b1da686-878a-462c-b2c6-a9690199939c",
     "vendor": "Google",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/googlesearch/v0"
   }
 ]

pkg/huggingface/config/definitions.json → pkg/huggingface/v0/config/definitions.json

@@ -68,6 +68,8 @@
     "type": "CONNECTOR_TYPE_AI",
     "uid": "0255ef87-33ce-4f88-b9db-8897f8c17233",
     "vendor": "Hugging Face",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/huggingface/v0"
   }
 ]

pkg/instill/config/definitions.json → pkg/instill/v0/config/definitions.json

@@ -68,6 +68,8 @@
     "type": "CONNECTOR_TYPE_AI",
     "uid": "ddcf42c3-4c30-4c65-9585-25f1c89b2b48",
     "vendor": "Instill",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/instill/v0"
   }
 ]

pkg/integration_test.go

@@ -12,7 +12,7 @@ import (
 	"google.golang.org/protobuf/types/known/structpb"
 
 	"github.com/instill-ai/component/pkg/base"
-	"github.com/instill-ai/connector/pkg/openai"
+	"github.com/instill-ai/connector/pkg/openai/v0"
 	"github.com/instill-ai/x/errmsg"
 )
 

pkg/main.go

@@ -8,20 +8,20 @@ import (
 	"google.golang.org/protobuf/types/known/structpb"
 
 	"github.com/instill-ai/component/pkg/base"
-	"github.com/instill-ai/connector/pkg/airbyte"
-	"github.com/instill-ai/connector/pkg/archetypeai"
-	"github.com/instill-ai/connector/pkg/bigquery"
-	"github.com/instill-ai/connector/pkg/googlecloudstorage"
-	"github.com/instill-ai/connector/pkg/googlesearch"
-	"github.com/instill-ai/connector/pkg/huggingface"
-	"github.com/instill-ai/connector/pkg/instill"
-	"github.com/instill-ai/connector/pkg/numbers"
-	"github.com/instill-ai/connector/pkg/openai"
-	"github.com/instill-ai/connector/pkg/pinecone"
-	"github.com/instill-ai/connector/pkg/redis"
-	"github.com/instill-ai/connector/pkg/restapi"
-	"github.com/instill-ai/connector/pkg/stabilityai"
-	"github.com/instill-ai/connector/pkg/website"
+	"github.com/instill-ai/connector/pkg/airbyte/v0"
+	"github.com/instill-ai/connector/pkg/archetypeai/v0"
+	"github.com/instill-ai/connector/pkg/bigquery/v0"
+	"github.com/instill-ai/connector/pkg/googlecloudstorage/v0"
+	"github.com/instill-ai/connector/pkg/googlesearch/v0"
+	"github.com/instill-ai/connector/pkg/huggingface/v0"
+	"github.com/instill-ai/connector/pkg/instill/v0"
+	"github.com/instill-ai/connector/pkg/numbers/v0"
+	"github.com/instill-ai/connector/pkg/openai/v0"
+	"github.com/instill-ai/connector/pkg/pinecone/v0"
+	"github.com/instill-ai/connector/pkg/redis/v0"
+	"github.com/instill-ai/connector/pkg/restapi/v0"
+	"github.com/instill-ai/connector/pkg/stabilityai/v0"
+	"github.com/instill-ai/connector/pkg/website/v0"
 
 	pipelinePB "github.com/instill-ai/protogen-go/vdp/pipeline/v1beta"
 )

pkg/numbers/config/definitions.json → pkg/numbers/v0/config/definitions.json

@@ -34,6 +34,8 @@
     "type": "CONNECTOR_TYPE_BLOCKCHAIN",
     "uid": "70d8664a-d512-4517-a5e8-5d4da81756a7",
     "vendor": "Numbers Protocol",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/numbers/v0"
   }
 ]

pkg/openai/config/definitions.json → pkg/openai/v0/config/definitions.json

@@ -43,6 +43,8 @@
     "type": "CONNECTOR_TYPE_AI",
     "uid": "9fb6a2cb-bff5-4c69-bc6d-4538dd8e3362",
     "vendor": "OpenAI",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/openai/v0"
   }
 ]

pkg/pinecone/config/definitions.json → pkg/pinecone/v0/config/definitions.json

@@ -43,6 +43,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "4b1dcf82-e134-4ba7-992f-f9a02536ec2b",
     "vendor": "Pinecone",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/pinecone/v0"
   }
 ]

pkg/redis/config/definitions.json → pkg/redis/v0/config/definitions.json

@@ -154,6 +154,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "fd0ad325-f2f7-41f3-b247-6c71d571b1b8",
     "vendor": "Redis Labs",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/redis/v0"
   }
 ]

pkg/restapi/config/definitions.json → pkg/restapi/v0/config/definitions.json

@@ -159,6 +159,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "5ee55a5c-6e30-4c7a-80e8-90165a729e0a",
     "vendor": "",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/restapi/v0"
   }
 ]

pkg/stabilityai/config/definitions.json → pkg/stabilityai/v0/config/definitions.json

@@ -35,6 +35,8 @@
     "type": "CONNECTOR_TYPE_AI",
     "uid": "c86a95cc-7d32-4e22-a290-8c699f6705a4",
     "vendor": "Stability AI",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/stabilityai/v0"
   }
 ]

pkg/website/config/definitions.json → pkg/website/v0/config/definitions.json

@@ -23,6 +23,8 @@
     "type": "CONNECTOR_TYPE_DATA",
     "uid": "98909958-db7d-4dfe-9858-7761904be17e",
     "vendor": "",
-    "vendor_attributes": {}
+    "vendor_attributes": {},
+    "version": "0.1.0-alpha",
+    "source_url": "github.com/instill-ai/connector/blob/main/pkg/website/v0"
   }
 ]