Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 1.0 branch into main #161

Merged
merged 147 commits into from
Apr 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
147 commits
Select commit Hold shift + click to select a range
b8584d4
Use datadog gem v2, ruby 2.7 is minimal supported version now
anmarchenko Jan 29, 2024
c5e22d6
datadog_cov native extension
anmarchenko Mar 14, 2024
b85198d
test datadog_cov extension
anmarchenko Mar 14, 2024
1df52e0
fix bool type compilation error
anmarchenko Mar 14, 2024
cdc74cb
exclude native binary artifacts from gemspec
anmarchenko Mar 14, 2024
31f46a4
don't run code coverage specs on JRuby
anmarchenko Mar 14, 2024
452812e
do not require native extension on JRuby
anmarchenko Mar 14, 2024
0db6f74
try again
anmarchenko Mar 14, 2024
987ccd0
don't use class name in top-level describe
anmarchenko Mar 14, 2024
c97d3f3
ITR::Runner has thread local coverage collector
anmarchenko Mar 14, 2024
abf11af
gemfile updates
anmarchenko Mar 14, 2024
0ba3e9c
ITR code coverage start/stop in Recorder
anmarchenko Mar 14, 2024
90f4e61
freeze source file name to reduce object allocations
anmarchenko Mar 15, 2024
23628ad
frozen strings do not make it faster
anmarchenko Mar 15, 2024
15083dc
require native extension on ITR configuration step
anmarchenko Mar 15, 2024
c8e8d93
add Datadog::CI::Cov placeholder
anmarchenko Mar 15, 2024
8d60f52
fix jruby tests
anmarchenko Mar 15, 2024
466a313
multithreaded code coverage and mixins tests
anmarchenko Mar 15, 2024
284cdbd
note that UTF-8 paths are not supported yet, will be added as an opt-…
anmarchenko Mar 15, 2024
270b349
specs for ITR::Runner#start_coverage
anmarchenko Mar 15, 2024
d1a1d27
add more specs, try to fix release spec with forcing encoding for git…
anmarchenko Mar 15, 2024
ef9861f
exclude cov_spec in JRUby completely
anmarchenko Mar 15, 2024
ddf9dbb
move cov tests and skip them completely on JRuby
anmarchenko Mar 15, 2024
109ad1e
do not compile native extension on unsupported platforms
anmarchenko Mar 20, 2024
caa5f15
check that dd_cov_data->root pointer is not NULL to avoid segfault
anmarchenko Mar 20, 2024
116c38e
minor fixes
anmarchenko Mar 20, 2024
a71bf38
store root as RString; convert to char* before using it for prefix check
anmarchenko Mar 20, 2024
5b25390
initialize dd_cov_data->mode field when allocating dd_cov_data struct
anmarchenko Mar 20, 2024
554a148
use Queue to synchronize threads in multithreaded coverage spec
anmarchenko Mar 20, 2024
74efff3
add a simple test for lines coverage
anmarchenko Mar 20, 2024
0a3d0c7
store covered lines in hash
anmarchenko Mar 20, 2024
6f38ac6
use ruby 2.7 syntax in ITR::Runner
anmarchenko Mar 20, 2024
1382d89
add gem datadog explicitly to integration test's Gemfile to make test…
anmarchenko Mar 20, 2024
af8ad9d
filter out invalid line numbers when tracking lines coverage
anmarchenko Mar 21, 2024
b93bfbd
use RSTRING_LEN to avoid computing root's length on every event
anmarchenko Mar 21, 2024
ecc669e
mark dd_cov_data->root as movable and update pointer on compaction
anmarchenko Mar 21, 2024
782ea8f
move coverage collector to Datadog::CI::ITR::Coverage namespace as it…
anmarchenko Mar 21, 2024
3c9d84f
fix ITR::Runner type signature
anmarchenko Mar 21, 2024
df154f3
add dummy transport for code coverage events
anmarchenko Mar 21, 2024
047b83c
citestcov intake support for api
anmarchenko Mar 21, 2024
d1308cb
fix type signatures for events
anmarchenko Mar 21, 2024
e787d68
finish multipart payload construction
anmarchenko Mar 22, 2024
28e50b3
serialize coverage event to MessagePack
anmarchenko Mar 22, 2024
0c274ee
Add Coverage::Transport for encoding and sending out the coverage events
anmarchenko Mar 22, 2024
65628d9
create EventPlatformTransport to hold common EvP client functionality
anmarchenko Mar 22, 2024
c60ad0e
TestVisibility::Transport is now a subclass of EventPlatformTransport…
anmarchenko Mar 22, 2024
a74369a
add gem datadog v2.0.0.beta1 to gemspec
anmarchenko Mar 25, 2024
a619bfb
Update Changelog
anmarchenko Mar 25, 2024
4cd18d8
Bump version 1.0.0.beta1
anmarchenko Mar 25, 2024
c4c27ed
update lockfiles
anmarchenko Mar 25, 2024
6f0602e
change README.md with instructions to setup this gem instead of ddtrace
anmarchenko Mar 25, 2024
256dd53
put the correct gem in integration tests gemfile
anmarchenko Mar 25, 2024
81214e8
add ITR::Coverage::Writer which encapsulates a background worker thre…
anmarchenko Mar 26, 2024
4a2be30
configure coverage writer for ITR
anmarchenko Mar 26, 2024
af919df
convert coverage file paths to be relative to the git root
anmarchenko Mar 26, 2024
8ec9bb9
print coverage events in debug mode
anmarchenko Mar 27, 2024
c84c2e7
add ITR::Runner#shutdown! method that is being called within Core::Co…
anmarchenko Mar 27, 2024
e0ad123
fix NullRecorder type signature and method parameters compatibility w…
anmarchenko Mar 27, 2024
55707bf
fix null recorder spec
anmarchenko Mar 27, 2024
6584d15
additional specs
anmarchenko Mar 27, 2024
c609fe1
skip #stop_coverage spec on JRuby
anmarchenko Mar 27, 2024
455722f
add some specs for coverage writer and fix #work_pending? method beha…
anmarchenko Mar 27, 2024
b3087cb
more thorough testing of Coverage::Writer
anmarchenko Mar 28, 2024
03385fa
add CoverageHelpers module
anmarchenko Mar 28, 2024
db670df
test rspec instrumentation with code coverage collection
anmarchenko Mar 28, 2024
af88c5d
setup test coverage writer only in CI mode helpers
anmarchenko Apr 2, 2024
68ddf33
skip coverage tests in JRuby
anmarchenko Apr 2, 2024
08affb0
test code coverage collection for rspec instrumentation
anmarchenko Apr 2, 2024
03e00de
test instrumentations with code coverage enabled;
anmarchenko Apr 2, 2024
a6a8766
skip coverage tests on jruby
anmarchenko Apr 2, 2024
70cfeeb
add static analysis config by DD
anmarchenko Apr 4, 2024
bf8177c
fix release_gem_spec
anmarchenko Apr 4, 2024
6ed6519
add Datadog::CI::Git::LocalRepository module
anmarchenko Apr 4, 2024
44a8ce6
migrate git commands from Utils::Git to Git::LocalRepository
anmarchenko Apr 4, 2024
e76f662
make exec_git_command private for security reasons
anmarchenko Apr 4, 2024
3d5ab3b
add LocalRepository.git_commits method to fetch 1000 latest commits
anmarchenko Apr 4, 2024
4a37f17
add git_commits_rev_list method to fetch git commits tree to upload
anmarchenko Apr 4, 2024
cbd63f1
add git_generate_packfiles method
anmarchenko Apr 5, 2024
a4b8eb9
skip packfiles testing for jruby for now
anmarchenko Apr 5, 2024
276eb41
test for git_tag method
anmarchenko Apr 5, 2024
d005644
if commit_tree is nil then git_generate_packfiles returns nil to indi…
anmarchenko Apr 5, 2024
638c0c4
Git::LocalRepository.repository_name uses log_failure instead of call…
anmarchenko Apr 5, 2024
8f69629
add Git::SearchCommits callable that sends backend API request to che…
anmarchenko Apr 5, 2024
334c112
LocalRepository.git_generate_packfiles checks that path is a real fil…
anmarchenko Apr 8, 2024
7a72236
filter invalid commits to avoid shell injections
anmarchenko Apr 8, 2024
830801f
Git::UploadPackfile client to send packfile to the gitdb backend
anmarchenko Apr 8, 2024
5023f9c
jruby produce a different error message for non existing file
anmarchenko Apr 8, 2024
bd8098a
first draft of Git::TreeUploader
anmarchenko Apr 8, 2024
115192f
make latest_commit/head_commit check shorter
anmarchenko Apr 8, 2024
8a10105
throwing more stuff into the fire
anmarchenko Apr 8, 2024
f53b5ce
return packfiles prefix from LocalRepository.git_generate_packfiles
anmarchenko Apr 9, 2024
2e03864
add Packfiles module that encapsulates generating git packfiles in so…
anmarchenko Apr 9, 2024
721276e
add git tree uploader service
anmarchenko Apr 9, 2024
5af1983
some more debug logging for tree uploader
anmarchenko Apr 9, 2024
e6e0be8
fix tests failed because of logging
anmarchenko Apr 9, 2024
53471af
add git_shallow_clone? method to LocalRepository module
anmarchenko Apr 9, 2024
a5fb173
test .git_shallow_clone? method with a reals shallow clone
anmarchenko Apr 9, 2024
40570de
test .git_commits with shallow clone
anmarchenko Apr 9, 2024
b22e10f
add .git_unshallow method
anmarchenko Apr 9, 2024
b1742da
fix jruby specs
anmarchenko Apr 9, 2024
4e7f4c5
add repository unshallowing logic to Git::TreeUploader
anmarchenko Apr 9, 2024
9c2ce30
actually the unshallowing logic is the other way around: unshallow wh…
anmarchenko Apr 9, 2024
3a06eb7
remove unneeded code
anmarchenko Apr 9, 2024
e03d8b9
better naming, replace filter_map with filter, better tests for .is_s…
anmarchenko Apr 10, 2024
dc0ab2c
added retries in case if git command returns nil status (breoken pipe?)
anmarchenko Apr 10, 2024
80e8f65
add Datadog::Core::Worker sigs
anmarchenko Apr 10, 2024
ab35606
Git::Packfiles do not fail if tmp folder does not exist
anmarchenko Apr 10, 2024
a560ce8
log when git tree upload finishes
anmarchenko Apr 10, 2024
5f8a835
add setting for git_metadata_upload_enabled
anmarchenko Apr 10, 2024
30cfdef
set git_metadata_upload_enabled to false by default during tests
anmarchenko Apr 10, 2024
0dbe384
add async worker that uploads git tree information after test session…
anmarchenko Apr 10, 2024
1f16ee6
improve type signatures for git upload worker
anmarchenko Apr 10, 2024
c22196e
test git command retry logic
anmarchenko Apr 11, 2024
d0391a4
worker specs
anmarchenko Apr 11, 2024
2e4e0fb
test that Recorder starts git metadata upload when test session starts
anmarchenko Apr 11, 2024
ed6f73e
better thread synchronization in Worker spec
anmarchenko Apr 11, 2024
e3a5742
add #require_git? method to RemoteSettingsApi::Response
anmarchenko Apr 11, 2024
de5d48a
wait for git upload and request library configuration again if requir…
anmarchenko Apr 11, 2024
1cc48aa
add skippable tests api client
anmarchenko Apr 12, 2024
858c81a
add additional logging for the case where git metadata upload was not…
anmarchenko Apr 12, 2024
6f9e083
request skippable tests when configuring ITR
anmarchenko Apr 12, 2024
7ffaa3f
fix failing tests
anmarchenko Apr 15, 2024
06a4a1b
report os.version tag for all CI events, fix remote settings request …
anmarchenko Apr 15, 2024
a76c311
add #os_version method to Datadog::CI::Span
anmarchenko Apr 15, 2024
3b4a6b9
send os.version in library settings and skippable tests API requests
anmarchenko Apr 15, 2024
c62c0f0
do not allow to skip dd_env parameter anywhere
anmarchenko Apr 15, 2024
28cea6a
store skippable tests as Set for faster checks
anmarchenko Apr 15, 2024
d01e18b
add retries with exponential backoff on network errors
anmarchenko Apr 16, 2024
10dc249
test ITR::Runner state after skippable tests configuration
anmarchenko Apr 16, 2024
c794a19
minor change for skippable spec
anmarchenko Apr 16, 2024
b843562
mark tests as skipped by ITR if they are found in skippable set
anmarchenko Apr 17, 2024
fd57ee3
do not send coverage events for tests skipped by test framework or by…
anmarchenko Apr 17, 2024
95bada0
disabling code coverage if ITR wants to skip test might break coverag…
anmarchenko Apr 17, 2024
2cc4971
ITR increments skipped tests count only after test is finished to acc…
anmarchenko Apr 17, 2024
4179162
set ITR tags with number of skipped tests when session ends
anmarchenko Apr 17, 2024
79b9ead
skipping tests by ITR implementation for minitest
anmarchenko Apr 18, 2024
ab97de7
write specs until the end
anmarchenko Apr 18, 2024
3393eff
rename skipped by itr tag
anmarchenko Apr 18, 2024
5a36fe9
additionally use test.parameters when determining if test is skippable
anmarchenko Apr 19, 2024
b476d8e
rspec support for ITR skipping
anmarchenko Apr 19, 2024
96183f0
trying out approaches for test skipping support in cucumber
anmarchenko Apr 19, 2024
75c70e2
finally working prototype
anmarchenko Apr 19, 2024
cd6ee1b
fix failing tests with datadog 2.0.0.beta2 and update datadog dependency
anmarchenko Apr 22, 2024
a6dbf60
temporarily skip shallow clone tests
anmarchenko Apr 22, 2024
e31a740
cucumber test skipping implementation done
anmarchenko Apr 22, 2024
b26de0f
rename TestRun.test_full_name to TestRun.skippable_test_id
anmarchenko Apr 22, 2024
314ab7e
update test dependencies
anmarchenko Apr 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,11 @@ Gemfile-*.lock
.ruby-version
.DS_Store
/test.rb

# Native extension binaries
lib/**/*.bundle
lib/**/*.so
lib/**/*.o
lib/**/*.dylib
lib/**/*.dll

4 changes: 2 additions & 2 deletions .standard.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
ruby_version: 2.1
ruby_version: 2.7
format: progress

ignore:
- 'gemfiles/**/*'
- 'tasks/**/*'
- 'yard/**/*'
- 'yard/**/*'
8 changes: 0 additions & 8 deletions .standard_todo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,6 @@
ignore:
- lib/datadog/ci/test_visibility/serializers/base.rb:
- Style/HashExcept
- lib/datadog/ci/contrib/minitest/integration.rb:
- Style/SafeNavigation
- lib/datadog/ci/contrib/cucumber/integration.rb:
- Style/SafeNavigation
- lib/datadog/ci/contrib/rspec/integration.rb:
- Style/SafeNavigation
- lib/datadog/ci/ext/environment.rb:
- Style/SafeNavigation
- spec/support/log_helpers.rb:
- Performance/UnfreezeString
- Appraisals:
Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
## [Unreleased]

## [1.0.0.beta1] - 2024-03-25

### Added

* datadog-cov native extension for per test code coverage ([#137])
* citestcov transport to serialize and send code coverage events ([#148])

### Removed

* Ruby 2.1-2.6 support is dropped

## [0.8.3] - 2024-03-20

### Fixed
Expand Down Expand Up @@ -188,6 +199,7 @@ Currently test suite level visibility is not used by our instrumentation: it wil
* Ruby versions < 2.7 no longer supported ([#8][])

[Unreleased]: https://github.com/DataDog/datadog-ci-rb/compare/v0.8.3...main
[1.0.0.beta1]: https://github.com/DataDog/datadog-ci-rb/compare/v0.8.3...v1.0.0.beta1
[0.8.3]: https://github.com/DataDog/datadog-ci-rb/compare/v0.8.2...v0.8.3
[0.8.2]: https://github.com/DataDog/datadog-ci-rb/compare/v0.8.1...v0.8.2
[0.8.1]: https://github.com/DataDog/datadog-ci-rb/compare/v0.8.0...v0.8.1
Expand Down Expand Up @@ -258,7 +270,9 @@ Currently test suite level visibility is not used by our instrumentation: it wil
[#123]: https://github.com/DataDog/datadog-ci-rb/issues/123
[#131]: https://github.com/DataDog/datadog-ci-rb/issues/131
[#134]: https://github.com/DataDog/datadog-ci-rb/issues/134
[#137]: https://github.com/DataDog/datadog-ci-rb/issues/137
[#139]: https://github.com/DataDog/datadog-ci-rb/issues/139
[#141]: https://github.com/DataDog/datadog-ci-rb/issues/141
[#142]: https://github.com/DataDog/datadog-ci-rb/issues/142
[#145]: https://github.com/DataDog/datadog-ci-rb/issues/145
[#148]: https://github.com/DataDog/datadog-ci-rb/issues/148
6 changes: 3 additions & 3 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ source "https://rubygems.org"
# Specify your gem's dependencies in datadog-ci.gemspec
gemspec

# needed to run tests, always present at runtime
gem "ddtrace"

gem "pry"
gem "rake"
gem "os"

# To compile native extensions
gem "rake-compiler"

gem "climate_control"

gem "rspec"
Expand Down
2 changes: 1 addition & 1 deletion LICENSE-3rdparty.csv
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Component,Origin,License,Copyright
dd-trace-rb,https://github.com/DataDog/dd-trace-rb,Apache 2.0,"Copyright 2016-Present Datadog, Inc."
datadog,https://github.com/DataDog/dd-trace-rb,Apache 2.0,"Copyright 2016-Present Datadog, Inc."
msgpack,https://rubygems.org/gems/msgpack,Apache-2.0,"Copyright (c) 2008-2015 Sadayuki Furuhashi"
12 changes: 3 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,14 @@
Datadog's Ruby Library for instrumenting your test and continuous integration pipeline.
Learn more on our [official website](https://docs.datadoghq.com/tests/) and check out our [documentation for this library](https://docs.datadoghq.com/tests/setup/ruby/?tab=cloudciprovideragentless).

> [!IMPORTANT]
> The `datadog-ci` gem is currently a component of [`ddtrace`](https://github.com/datadog/dd-trace-rb) and should not be used without it.
>
> We expect this to change in the future.

## Installation

Add to your Gemfile.

```ruby
gem "ddtrace"
group :test do
gem "datadog-ci"
end
```

## Usage
Expand Down Expand Up @@ -52,7 +49,6 @@ end
| --- | ----------- | ------- |
| `enabled` | Defines whether RSpec tests should be traced. Useful for temporarily disabling tracing. `true` or `false` | `true` |
| `service_name` | Service name used for `rspec` instrumentation. | `'rspec'` |
| `operation_name` | *DEPRECATED, to be removed in 1.0* Operation name used for `rspec` instrumentation (has no effect in agentless mode or when using newer agent versions). | `'rspec.example'` |

### Minitest

Expand Down Expand Up @@ -104,7 +100,6 @@ end
| --- | ----------- | ------- |
| `enabled` | Defines whether Minitest tests should be traced. Useful for temporarily disabling tracing. `true` or `false` | `true` |
| `service_name` | Service name used for `minitest` instrumentation. | `'minitest'` |
| `operation_name` | *DEPRECATED, to be removed in 1.0* Operation name used for `minitest` instrumentation (has no effect in agentless mode or when using newer agent versions). | `'minitest.test'` |

### Cucumber

Expand Down Expand Up @@ -135,7 +130,6 @@ end
| --- | ----------- | ------- |
| `enabled` | Defines whether Cucumber tests should be traced. Useful for temporarily disabling tracing. `true` or `false` | `true` |
| `service_name` | Service name used for `cucumber` instrumentation. | `'cucumber'` |
| `operation_name` | *DEPRECATED, to be removed in 1.0* Operation name used for `cucumber` instrumentation (has no effect in agentless mode or when using newer agent versions). | `'cucumber.test'` |

## Agentless mode

Expand Down
24 changes: 22 additions & 2 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ require_relative "lib/datadog/ci/version"
require "bundler/gem_tasks"
require "rspec/core/rake_task"
require "yard"
require "rake/extensiontask"

RSpec::Core::RakeTask.new(:spec)

Expand Down Expand Up @@ -107,8 +108,12 @@ end
namespace :spec do
desc "" # "Explicitly hiding from `rake -T`"
RSpec::Core::RakeTask.new(:main) do |t, args|
t.pattern = "spec/**/*_spec.rb"
t.exclude_pattern = "spec/**/{contrib}/**/*_spec.rb,"
t.pattern = if RUBY_ENGINE == "jruby"
"spec/datadog/**/*_spec.rb"
else
"spec/datadog/**/*_spec.rb,spec/ddcov/**/*_spec.rb"
end
t.exclude_pattern = "spec/datadog/**/{contrib}/**/*_spec.rb,"
t.rspec_opts = args.to_a.join(" ")
end

Expand All @@ -132,3 +137,18 @@ end

desc "CI task; it runs all tests for current version of Ruby"
task ci: "test:all"

# native extensions
Rake::ExtensionTask.new("datadog_cov.#{RUBY_VERSION}_#{RUBY_PLATFORM}") do |ext|
ext.ext_dir = "ext/datadog_cov"
end

task :compile_ext do
if RUBY_ENGINE == "ruby"
Rake::Task[:clean].invoke
Rake::Task[:compile].invoke
end
end

# run compile before any tests are run
Rake::Task["test:all"].prerequisite_tasks.each { |t| t.enhance([:compile_ext]) }
4 changes: 4 additions & 0 deletions Steepfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ target :lib do
library "minitest"
library "net-http"
library "zlib"
library "securerandom"
library "tmpdir"
library "fileutils"
library "socket"

repo_path "vendor/rbs"
library "ddtrace"
Expand Down
5 changes: 5 additions & 0 deletions datadog-ci.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,15 @@ Gem::Specification.new do |spec|
LICENSE*
NOTICE
README.md
ext/**/*
lib/**/*
]].select { |fn| File.file?(fn) } # We don't want directories, only files
.reject { |fn| fn.end_with?(".so", ".bundle") } # Exclude local native binary artifacts

spec.require_paths = ["lib"]

spec.add_dependency "datadog", "~> 2.0.0.beta2"
spec.add_dependency "msgpack"

spec.extensions = ["ext/datadog_cov/extconf.rb"]
end
Loading
Loading