-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CIVIS-2467] Support test visibility protocol via Datadog Agent with EVP proxy #51
Conversation
Codecov Report
@@ Coverage Diff @@
## main #51 +/- ##
==========================================
+ Coverage 99.15% 99.16% +0.01%
==========================================
Files 95 102 +7
Lines 3324 3487 +163
Branches 129 134 +5
==========================================
+ Hits 3296 3458 +162
- Misses 28 29 +1
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
…TP in TestVisibility::Transport
846b448
to
8dbbf5c
Compare
…at by agent in "endpoints" list
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code looks good.
Given this is a sizeable new feature, would it make sense if we had at least one integration test for it? (unless there's already one and I interpreted it as unit test)
@marcotc we run integration tests in https://github.com/DataDog/test-environment but this is a good point, I kind of neglected integration tests that we have in this repo I will add a task for myself to add more of them |
lib/datadog/ci/transport/api/base.rb
Outdated
def request(path:, payload:, verb: "post") | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could make the Base class take an Datadog::CI::Transport::HTTP
instance in the initialize method + whatever the child needs and move the logic to create the correct HTTP instance to the builder.
Since the request method seems to pass the arguments to the http request method that method is suited to be part of the base class.
And we can make sure that every child class of Base have its own headers
method by marking in the base as not implemented
Here is with code what I mean:
# frozen_string_literal: true
module Datadog
module CI
module Transport
module Api
class Base
attr_reader :http
def initialize(http:)
@http = http
end
def request(path:, payload:, verb: "post")
http.request(
path: path,
payload: payload,
verb: verb,
headers: headers
)
end
private
def headers
raise NotImplementedError
end
end
end
end
end
end
# frozen_string_literal: true
require_relative "base"
module Datadog
module CI
module Transport
module Api
class CIIntake < Base
attr_reader :api_key
def initialize(http:, api_key:)
super(http: http)
@api_key = api_key
end
private
def headers
{
Ext::Transport::HEADER_CONTENT_TYPE => Ext::Transport::CONTENT_TYPE_MESSAGEPACK
Ext::Transport::HEADER_DD_API_KEY => api_key
}
end
end
end
end
end
end
# frozen_string_literal: true
require_relative "base"
module Datadog
module CI
module Transport
module Api
class EVPProxy < Base
def request(path:, payload:, verb: "post")
path = "#{Ext::Transport::EVP_PROXY_PATH_PREFIX}#{path.sub(/^\//, "")}"
super(
path: path,
payload: payload,
verb: verb,
headers: headers
)
end
private
def container_id
return @container_id if defined?(@container_id)
@container_id = Datadog::Core::Environment::Container.container_id
end
def headers
headers = {
Ext::Transport::HEADER_CONTENT_TYPE => Ext::Transport::CONTENT_TYPE_MESSAGEPACK
Ext::Transport::HEADER_EVP_SUBDOMAIN => Ext::Transport::TEST_VISIBILITY_INTAKE_HOST_PREFIX
}
c_id = container_id
headers[Ext::Transport::HEADER_CONTAINER_ID] = c_id unless c_id.nil?
headers
end
end
end
end
end
end
# frozen_string_literal: true
require_relative "../http"
require_relative "ci_intake"
require_relative "evp_proxy"
module Datadog
module CI
module Transport
module Api
module Builder
def self.build_ci_test_cycle_api(settings)
dd_site = settings.site || Ext::Transport::DEFAULT_DD_SITE
url = settings.ci.agentless_url ||
"https://#{Ext::Transport::TEST_VISIBILITY_INTAKE_HOST_PREFIX}.#{dd_site}:443"
uri = URI.parse(url)
raise "Invalid agentless mode URL: #{url}" if uri.host.nil?
http = Datadog::CI::Transport::HTTP.new(
host: uri.host,
port: uri.port,
ssl: uri.scheme == "https" || uri.port == 443,
compress: true
)
CIIntake.new(http: http, api_key: settings.api_key)
end
def self.build_evp_proxy_api(agent_settings)
http = Datadog::CI::Transport::HTTP.new(
host: agent_settings.hostname,
port: agent_settings.port,
ssl: agent_settings.ssl,
timeout: agent_settings.timeout_seconds,
compress: false
)
EVPProxy.new(http: http)
end
end
end
end
end
end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, useful suggestion again @GustavoCaso, this can further decouple APIs from underlying transport used, will try to refactor it today
@anmarchenko if this feature is covered by https://github.com/DataDog/test-environment, then all good! |
module CI | ||
module Transport | ||
module Api | ||
class CIIntake < Base |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CIIntake
😛
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks ugly, I know :( EVPProxy looks not so much better
I think I should drop abbreviations and call them CiIntake and EvpProxy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The official name of the intake is CiTestCycle, I will use it
What does this PR do?
For the case when agentless mode is not configured, we check whether Datadog Agent has EVP (event platform) proxy endpoint and if it does, we use test visibility protocol via this proxy to submit traces to Datadog.
This PR introduces Datadog::CI::Transport::Api module that incapsulates API settings such as host, port, ssl usage, gzip compression, etc.
Datadog::CI::TestVisibility::Transport no longer constructs API layer directly: api gets injected in the transport on Components initialization phase.
Motivation
Implementing the next feature for CI visibility library on a road to GA.
How to test the change?
Run ruby tests locally without DD_CIVISIBLITY_AGENTLESS_ENABLED