Skip to content

Commit

Permalink
feat(cli): promote kitless
Browse files Browse the repository at this point in the history
  • Loading branch information
squakez committed Aug 29, 2024
1 parent 6935636 commit 1574fed
Show file tree
Hide file tree
Showing 19 changed files with 245 additions and 410 deletions.
8 changes: 5 additions & 3 deletions docs/modules/ROOT/pages/running/camel-runtimes.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
= Camel Runtimes (aka "sourceless" Integrations)

Camel K can run any runtime available in Apache Camel. However, this is possible only when the Camel application was previously built and packaged into a container image. Also, if you run through this option, some of the features offered by the operator may not be available. For example, you won't be able to discover Camel capabilities because the source is not available to the operator but embedded in the container image.
Camel K operator is traditionally in charge to perform a build from a Camel DSL source. The resulting Integration depends directly on an IntegrationKit, which is a reusable custom resource backing the final container image that your application will run. In the last versions, the only runtime the operator can build is Camel Quarkus (via Camel K Runtime project).

However Camel K can run any runtime available in Apache Camel. This is possible only when the Camel application was previously built and packaged into a container image externally. Mind that if you run through this option, some of the features offered by the operator may not be available. For example, you won't be able to discover Camel capabilities because the source is not available to the operator but embedded in the container image.

This option is quite interesting if in general you're building your applications externally, ie, via a CICD technology, and you want to delegate the operator only the "operational" part, taking care on your own of the building and publishing part.

Expand All @@ -15,7 +17,7 @@ You can have your own Camel application or just create a basic one for the purpo

The step above is a very quick way to create a basic Camel application in any of the available runtime. Let's imagine we've done this for Camel Main or we have already a Camel application as a Maven project. As the build part is something we want to take care on our own, we create a pipeline to build, containerize and push the container to a registry (see as a reference https://github.com/tektoncd/catalog/blob/main/task/kamel-run/0.1/samples/run-external-build.yaml[Camel K Tekton example]).

At this stage we do have a container image with our Camel application. We can use the `kamel` CLI to run our Camel application via `kamel run --image docker.io/my-org/my-app:1.0.0` tuning, if it's the case, with any of the trait or configuration required. Mind that, when you run an Integration with this option, the operator will create a **synthetic** IntegrationKit.
At this stage we do have a container image with our Camel application. We can use the `kamel` CLI to run our Camel application via `kamel run --image docker.io/my-org/my-app:1.0.0` tuning, if it's the case, with any of the trait or configuration required. As there is no creation of an IntegrationKit, this is also known as 'kit-less' Integration.

If all is good, in a few seconds (there is no build involved) you should have your application up and running and you can monitor and operate with Camel K as usual.

Expand All @@ -27,4 +29,4 @@ Every Camel application requires a `CamelCatalog` object to know how to perform
[[traits]]
== Trait configuration

Certain Camel K operational aspect may be driven by traits. When you're building the application outside the operator, some of those traits may not be executed as they are executed during the building phase that we are skipping when running **sourceless Integrations**. There is also no possible way to auto-tune certain traits that require the presence of the source. In this case, you should instead provide a trait configuration with the values that are required by your Integration (for example, Knative, Service and other deployment traits).
Certain Camel K operational aspect may be driven by traits. When you're building the application outside the operator, some of those traits may not be executed as they are executed during the building phase that we are skipping when running **sourceless Integrations**. There is also no possible way to auto-tune certain traits that require the presence of the source. In this case, you should instead provide a trait configuration with the values that are required by your Integration (for example, Knative, Service and other deployment traits).
10 changes: 1 addition & 9 deletions docs/modules/ROOT/pages/running/promoting.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,11 @@

As soon as you have an Integration running in your cluster, you will be challenged to move that Integration to an higher environment. Ie, you can test your Integration in a **development** environment, and, as soon as you're happy with the result, you will need to move it into a **production** environment.

== External IntegrationKits

When you create an Integration, the operator takes care to use an existing IntegrationKit or creating one from scratch. When you're moving your Integration across environments you'll therefore need to create an IntegrationKit accordingly. The creation of the IntegrationKit, in this case, is a simple copy of the original IntegrationKit with the `.spec.image` coming from the `.status.image` of the original IntegrationKit. Additionally, it has to be labelled as `camel.apache.org/kit.type: external`.

NOTE: these two configuration are required to avoid the new IntegrationKit to kick off a new build operation.

The copy is something you can do with any external tooling or using the `kamel promote` command as provided below:

== CLI `promote` command

Camel K has an opinionated way to achieve the promotion goal through the usage of `kamel promote` command. With this command you will be able to easily move an Integration from one namespace to another without worrying about any low level detail such as resources needed by the Integration. You only need to make sure that both the source operator and the destination operator are using the same container registry and that the destination namespace provides the required Configmaps, Secrets or Kamelets required by the Integration.

NOTE: in order to use the same container registry, you can use the `--registry` option during installation phase or change the IntegrationPlatform to reflect that accordingly.
NOTE: use dry run option (`-o yaml`) and export the result to any separated cluster or Git repository to perform a GitOps strategy.

Let's see a simple Integration that uses a Configmap to expose some message on an HTTP endpoint. We can start creating such an Integration and testing in a namespace called `development`:

Expand Down
5 changes: 0 additions & 5 deletions e2e/advanced/tekton_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,6 @@ func TestTektonLikeBehavior(t *testing.T) {
g.Eventually(IntegrationPodPhase(t, ctx, ns, name), TestTimeoutLong).Should(Equal(corev1.PodRunning))
g.Eventually(IntegrationConditionStatus(t, ctx, ns, name, v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
g.Eventually(IntegrationLogs(t, ctx, ns, name), TestTimeoutShort).Should(ContainSubstring("Echo"))
tektonIntegrationKitName := IntegrationKit(t, ctx, ns, name)()
g.Expect(tektonIntegrationKitName).ToNot(BeEmpty())
tektonIntegrationKitImage := KitImage(t, ctx, ns, tektonIntegrationKitName)()
g.Expect(tektonIntegrationKitImage).ToNot(BeEmpty())
g.Eventually(Kit(t, ctx, ns, tektonIntegrationKitName)().Status.Image, TestTimeoutShort).Should(Equal(externalImage))
})
})
}
10 changes: 6 additions & 4 deletions e2e/install/upgrade/upgrade_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ func TestUpgrade(t *testing.T) {
// Check the IntegrationPlatform has been reconciled
g.Eventually(PlatformPhase(t, ctx, ns)).Should(Equal(v1.IntegrationPlatformPhaseReady))
g.Eventually(PlatformVersion(t, ctx, ns)).Should(Equal(lastVersion))
lastRuntimeVersion := PlatformRuntimeVersion(t, ctx, ns)()

// We need a different namespace from the global operator
WithNewTestNamespace(t, func(ctx context.Context, g *WithT, nsIntegration string) {
Expand Down Expand Up @@ -117,6 +118,7 @@ func TestUpgrade(t *testing.T) {
// Check the IntegrationPlatform has been reconciled
g.Eventually(PlatformPhase(t, ctx, ns), TestTimeoutMedium).Should(Equal(v1.IntegrationPlatformPhaseReady))
g.Eventually(PlatformVersion(t, ctx, ns), TestTimeoutMedium).Should(Equal(defaults.Version))
g.Eventually(PlatformRuntimeVersion(t, ctx, ns), TestTimeoutMedium).Should(Equal(defaults.DefaultRuntimeVersion))

// Check the Integration hasn't been upgraded
g.Consistently(IntegrationVersion(t, ctx, nsIntegration, name), 15*time.Second, 3*time.Second).
Expand All @@ -135,14 +137,14 @@ func TestUpgrade(t *testing.T) {
g.Eventually(IntegrationVersion(t, ctx, nsIntegration, name), TestTimeoutMedium).Should(Equal(defaults.Version))

// Check the previous kit is not garbage collected
g.Eventually(Kits(t, ctx, ns, KitWithVersion(lastVersion))).Should(HaveLen(1))
g.Eventually(Kits(t, ctx, ns, KitWithRuntimeVersion(lastRuntimeVersion))).Should(HaveLen(1))
// Check a new kit is created with the current version
g.Eventually(Kits(t, ctx, ns, KitWithVersion(defaults.Version))).Should(HaveLen(1))
g.Eventually(Kits(t, ctx, ns, KitWithRuntimeVersion(defaults.DefaultRuntimeVersion))).Should(HaveLen(1))
// Check the new kit is ready
g.Eventually(Kits(t, ctx, ns, KitWithVersion(defaults.Version), KitWithPhase(v1.IntegrationKitPhaseReady)),
g.Eventually(Kits(t, ctx, ns, KitWithRuntimeVersion(defaults.DefaultRuntimeVersion), KitWithPhase(v1.IntegrationKitPhaseReady)),
TestTimeoutMedium).Should(HaveLen(1))

kit := Kits(t, ctx, ns, KitWithVersion(defaults.Version))()[0]
kit := Kits(t, ctx, ns, KitWithRuntimeVersion(defaults.DefaultRuntimeVersion))()[0]

// Check the Integration uses the new image
g.Eventually(IntegrationKit(t, ctx, nsIntegration, name), TestTimeoutMedium).Should(Equal(kit.Name))
Expand Down
22 changes: 12 additions & 10 deletions e2e/support/test_support.go
Original file line number Diff line number Diff line change
Expand Up @@ -1291,18 +1291,10 @@ func KitWithPhase(phase v1.IntegrationKitPhase) KitFilter {
}
}

func KitWithVersion(version string) KitFilter {
func KitWithRuntimeVersion(version string) KitFilter {
return &kitFilter{
filter: func(kit *v1.IntegrationKit) bool {
return kit.Status.Version == version
},
}
}

func KitWithVersionPrefix(versionPrefix string) KitFilter {
return &kitFilter{
filter: func(kit *v1.IntegrationKit) bool {
return strings.HasPrefix(kit.Status.Version, versionPrefix)
return kit.Status.RuntimeVersion == version
},
}
}
Expand Down Expand Up @@ -2124,6 +2116,16 @@ func PlatformVersion(t *testing.T, ctx context.Context, ns string) func() string
}
}

func PlatformRuntimeVersion(t *testing.T, ctx context.Context, ns string) func() string {
return func() string {
p := Platform(t, ctx, ns)()
if p == nil {
return ""
}
return p.Status.Build.RuntimeVersion
}
}

func PlatformPhase(t *testing.T, ctx context.Context, ns string) func() v1.IntegrationPlatformPhase {
return func() v1.IntegrationPlatformPhase {
p := Platform(t, ctx, ns)()
Expand Down
7 changes: 6 additions & 1 deletion pkg/apis/camel/v1/integration_types_support.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package v1

import (
"fmt"
"regexp"
"strings"

corev1 "k8s.io/api/core/v1"
Expand Down Expand Up @@ -88,9 +89,13 @@ func (in *Integration) UserDefinedSources() []SourceSpec {
return sources
}

var managedBuildKit = regexp.MustCompile("(.*)/(.*)/camel-k-kit-(.*)@sha256:(.*)")

// IsManagedBuild returns true when the Integration requires to be built by the operator.
func (in *Integration) IsManagedBuild() bool {
return in.Spec.Traits.Container == nil || in.Spec.Traits.Container.Image == ""
return in.Spec.Traits.Container == nil ||
in.Spec.Traits.Container.Image == "" ||
managedBuildKit.Match([]byte(in.Spec.Traits.Container.Image))
}

func (in *IntegrationSpec) AddSource(name string, content string, language Language) {
Expand Down
24 changes: 24 additions & 0 deletions pkg/apis/camel/v1/integration_types_support_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
"fmt"
"testing"

"github.com/apache/camel-k/v2/pkg/apis/camel/v1/trait"
"github.com/stretchr/testify/assert"
)

Expand Down Expand Up @@ -101,3 +102,26 @@ func TestGetConfigurationProperty(t *testing.T) {
v6 := integration.GetConfigurationProperty("key6")
assert.Equal(t, "", v6)
}

func TestManagedBuild(t *testing.T) {
integration := Integration{
Spec: IntegrationSpec{},
}
assert.True(t, integration.IsManagedBuild())
integration.Spec.Traits = Traits{
Container: &trait.ContainerTrait{},
}
assert.True(t, integration.IsManagedBuild())
integration.Spec.Traits = Traits{
Container: &trait.ContainerTrait{
Image: "registry.io/my-org/my-image",
},
}
assert.False(t, integration.IsManagedBuild())
integration.Spec.Traits = Traits{
Container: &trait.ContainerTrait{
Image: "10.100.107.57/camel-k/camel-k-kit-cr82ehho23os73cgua70@sha256:13e5a67d37665710c0bdd89701c7ae10aee393b00f5e4e09dc8ecc234763e7c2",
},
}
assert.True(t, integration.IsManagedBuild())
}
7 changes: 2 additions & 5 deletions pkg/apis/camel/v1/integrationkit_types_support.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ package v1
import (
"fmt"
"path/filepath"
"sort"
"strconv"

"github.com/apache/camel-k/v2/pkg/util/sets"
Expand Down Expand Up @@ -211,16 +210,14 @@ func (in *IntegrationKitStatus) GetConditions() []ResourceCondition {
}

// GetDependenciesPaths returns the set of dependency paths.
func (in *IntegrationKitStatus) GetDependenciesPaths() []string {
func (in *IntegrationKitStatus) GetDependenciesPaths() *sets.Set {
s := sets.NewSet()
for _, dep := range in.Artifacts {
path := filepath.Dir(dep.Target)
s.Add(fmt.Sprintf("%s/*", path))
}
values := s.List()
sort.Strings(values)

return values
return s
}

func (c IntegrationKitCondition) GetType() string {
Expand Down
13 changes: 8 additions & 5 deletions pkg/apis/camel/v1/integrationkit_types_support_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ limitations under the License.
package v1

import (
"sort"
"testing"

"github.com/stretchr/testify/assert"
Expand All @@ -39,9 +40,11 @@ func TestGetKitDependenciesDirectories(t *testing.T) {
},
}
paths := kit.Status.GetDependenciesPaths()
assert.Len(t, paths, 4)
assert.Equal(t, "my-dir/*", paths[0])
assert.Equal(t, "my-dir1/lib/*", paths[1])
assert.Equal(t, "my-dir1/lib2/*", paths[2])
assert.Equal(t, "my-dir2/lib/*", paths[3])
pathsArray := paths.List()
sort.Strings(pathsArray)
assert.Len(t, pathsArray, 4)
assert.Equal(t, "my-dir/*", pathsArray[0])
assert.Equal(t, "my-dir1/lib/*", pathsArray[1])
assert.Equal(t, "my-dir1/lib2/*", pathsArray[2])
assert.Equal(t, "my-dir2/lib/*", pathsArray[3])
}
Loading

0 comments on commit 1574fed

Please sign in to comment.