Skip to content
This repository has been archived by the owner on Jan 9, 2025. It is now read-only.

feat: set component versions to 0.1.0-alpha #123

Merged
merged 3 commits into from
Feb 9, 2024

Conversation

jvallesm
Copy link
Collaborator

@jvallesm jvallesm commented Feb 7, 2024

Because

  • We need a way to introduce breaking changes in a sensible way.
  • Component page will need the version and release stage of components.

This commit

  • Adds a version field to the component definition. Initial version is
    0.1.0-alpha as we don't have a way to migrate components at the moment and
    thus we can't guarantee that we won't introduce more breaking changes.
  • Moves each component package to a v0 directory so more than one major
    version can coexist.
  • Introduces the source_url field in the component definition. This field is
    required by the component page and will be returned in the connector /
    operator definition list.

Notes

Adding the source_url field in the definition isn't great as it references a
particular branch (in the blob/main bit) and because writing it by hand
shouldn't be required. It would be great if we could just build it from the
component ID but several connectors (e.g. gcp or instill-model) don't match
the package and the ID. Also, there's no clean way to build this value and
inject it into the Load[Connector|Operator]Definitions function.

To make the definitions file easier to build and less error-prone, we should
introduce a linter and perhaps a boilerplate builder in the future.

🤖 Script to migrate to v0

#!/usr/bin/env bash

set -eo pipefail

# This script modifies the directory of each component and sets their version
# to 0.1.0-alpha. It automatically updates the references to that package.
#
# Usage: sh newversion.sh

for p in $(fd definitions.json | cut -d '/' -f1-2 | sort | uniq); do

  # Move component into v0 subdirectory
  mkdir v0
  mv $p/* v0
  mv v0 $p/v0

  # Update references
  escaped=$(echo $p | sed 's/\//\\\//g')
  grep -rl github.com/instill-ai/connector/$p | sort | uniq | xargs -I '{}' sed -i '' "s/\(github.com\/instill-ai\/connector\/$escaped\)/\1\/v0/g" {}

  # Add version and source URL to definition
  v=0.1.0-alpha
  url=github.com/instill-ai/connector/blob/main/$p/v0

  jq ".[0] += {version: \"$v\", source_url: \"$url\"}" $p/v0/config/definitions.json > def.json
  mv def.json $p/v0/config/definitions.json

done;

@jvallesm jvallesm self-assigned this Feb 7, 2024
Copy link

linear bot commented Feb 7, 2024

Copy link

codecov bot commented Feb 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7059b5b) 62.18% compared to head (de14d32) 62.18%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #123   +/-   ##
=======================================
  Coverage   62.18%   62.18%           
=======================================
  Files          18       18           
  Lines        1489     1489           
=======================================
  Hits          926      926           
  Misses        440      440           
  Partials      123      123           
Flag Coverage Δ
unittests 62.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jvallesm jvallesm marked this pull request as ready for review February 7, 2024 10:55
@jvallesm
Copy link
Collaborator Author

jvallesm commented Feb 7, 2024

✅ QA

  • Created some pipelines with different connectors & tested existing pipelines
  • Checked connector definition list:
curl --request GET \
     --url 'localhost:8080/vdp/v1beta/connector-definitions?view=VIEW_BASIC' \
     --header 'accept: application/json' \
  --header 'Authorization: Bearer $INSTILL_KEY'  | jq | head -50
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11330    0 11330    0     0   191k      0 --:--:-- --:--:-- --:--:--  194k
{
  "connector_definitions": [
    {
      "name": "connector-definitions/stability-ai",
      "uid": "c86a95cc-7d32-4e22-a290-8c699f6705a4",
      "id": "stability-ai",
      "title": "Stability AI",
      "documentation_url": "https://www.instill.tech/docs/latest/vdp/ai-connectors/stability-ai",
      "icon": "Stability AI/stabilityai.svg",
      "spec": null,
      "type": "CONNECTOR_TYPE_AI",
      "tombstone": false,
      "public": true,
      "custom": false,
      "vendor": "Stability AI",
      "vendor_attributes": null,
      "source_url": "github.com/instill-ai/connector/blob/main/pkg/stabilityai/v1",
      "version": "1.0.0-alpha",
      "tasks": [
        {
          "name": "TASK_TEXT_TO_IMAGE",
          "title": "Text To Image",
          "description": "Generate a new image from a text prompt."
        },
        {
          "name": "TASK_IMAGE_TO_IMAGE",
          "title": "Image To Image",
          "description": "Modify an image based on a text prompt."
        }
      ]
    },
    {
      "name": "connector-definitions/instill-model",
      "uid": "ddcf42c3-4c30-4c65-9585-25f1c89b2b48",
      "id": "instill-model",
      "title": "Instill Model",
      "documentation_url": "https://www.instill.tech/docs/latest/vdp/ai-connectors/instill-model",
      "icon": "Instill AI/instillmodel.svg",
      "spec": null,
      "type": "CONNECTOR_TYPE_AI",
      "tombstone": false,
      "public": true,
      "custom": false,
      "vendor": "Instill",
      "vendor_attributes": null,
      "source_url": "github.com/instill-ai/connector/blob/main/pkg/instill/v1",
      "version": "1.0.0-alpha",
      "tasks": [
        {
          "name": "TASK_CLASSIFICATION",
      ...
}

Include changes to display new definition fields in the VIEW_BASIC view
in the component definition list
@jvallesm jvallesm force-pushed the jvalles/ins-3620-add-version-to-existing-components branch from 617e2da to 726325c Compare February 7, 2024 13:32
@jvallesm jvallesm changed the title feat: set component versions to 1.0.0-alpha feat: set component versions to 0.1.0-alpha Feb 8, 2024
@jvallesm jvallesm merged commit 81af1d5 into main Feb 9, 2024
14 checks passed
@jvallesm jvallesm deleted the jvalles/ins-3620-add-version-to-existing-components branch February 9, 2024 08:47
jvallesm added a commit to instill-ai/pipeline-backend that referenced this pull request Feb 9, 2024
Because

- PR instill-ai/connector#123 updated the
package path
  for airbyte connector, imported in `pipeline-backend`

This commit

- Updates the connector and operator versions and the airbyte path
donch1989 pushed a commit that referenced this pull request Feb 16, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.12.0-beta](v0.11.0-beta...v0.12.0-beta)
(2024-02-16)


### Features

* **restapi:** use `instillFormat: semi-structured/json` for request and
response body
([#126](#126))
([53606c1](53606c1))
* set component versions to 0.1.0-alpha
([#123](#123))
([81af1d5](81af1d5))
* store icons next to the component definition
([#122](#122))
([c67fc89](c67fc89))


### Bug Fixes

* **instill:** fix auth issue
([#128](#128))
([9908b3a](9908b3a))
* **pinecone:** fix issue that pinecone's icon padding is too big
([#127](#127))
([d27697e](d27697e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants