Skip to content

Commit

Permalink
Added a task to check for unpublished artifacts (#1453)
Browse files Browse the repository at this point in the history
Staged Publishing via Sonatype Nexus fail occasionally. Sometime, the
staging process was complete, sometimes not. Since subsequence
publish-jobs fail for previously properly published artifacts, it's not
easy to tell, which CI publish jobs in failure state need a restart.

Therefore you can use theis new command to check for already published
modules.

Example: Some modules where published, some not

In that case, you need to find that shard, which published these
modules. In the CI log, go to the last lines, where all published
modules are printed. If you found the correct shard, restart it.

```
> mill checkPublishedArtifacts --artifacts __.publishSelfDependency --version 3.0.0-M1-3-47512ee8 --ttl "1 sec"
...
1 targets failed
checkPublishedArtifacts Missing 46 of 276 published artifacts: 
- com.lihaoyi:ammonite-compiler-interface_3.3.0:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-compiler-interface_3.3.1:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-compiler-interface_3.3.2:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-compiler-interface_3.3.3:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-compiler_3.2.2:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-compiler_3.3.0:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-compiler_3.3.1:3.0.0-M1-3-47512ee8
...
- com.lihaoyi:ammonite-sshd_3.3.2:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-sshd_3.3.3:3.0.0-M1-3-47512ee8
- com.lihaoyi:ammonite-terminal_3:3.0.0-M1-3-47512ee8
```

Example: All modules published successfully

```
> mill show checkPublishedArtifacts --artifacts __.publishSelfDependency --version 3.0.0-M1
[1/1] show 
[1/1] show > [2485/2485] checkPublishedArtifacts 
"All artifacts published for version 3.0.0-M1"
```

Example: All modules not published

```
> mill show checkPublishedArtifacts --artifacts __.publishSelfDependency --version 3.0.0-M2
[1/1] show 
[1/1] show > [2485/2485] checkPublishedArtifacts 
1 targets failed
show 1 targets failed
checkPublishedArtifacts All artifacts missing for version 3.0.0-M2
```

Pull request: #1453
  • Loading branch information
lefou authored Mar 8, 2024
1 parent 9b5bf77 commit 8a463ab
Showing 1 changed file with 42 additions and 6 deletions.
48 changes: 42 additions & 6 deletions build.sc
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
// plugins
import $ivy.`com.lihaoyi::mill-contrib-bloop:$MILL_VERSION`
import $ivy.`io.get-coursier::coursier-launcher:2.1.0-RC1`
import $file.ci.upload

// imports
import java.util.concurrent.atomic.AtomicInteger
import scala.concurrent.{Await, ExecutionContext, Future, duration}
import scala.util.chaining.scalaUtilChainingOps
import coursier.mavenRepositoryString
import mill._
import scalalib._
import publish._
import mill.api.Result
import mill.contrib.bloop.Bloop
import mill.scalalib.api.ZincWorkerUtil._
import coursier.mavenRepositoryString
import mill.define.Command
import mill.main.Tasks
import mill.scalalib._
import mill.scalalib.publish._
import mill.scalalib.api.ZincWorkerUtil._
import mill.testrunner.TestRunner

import scala.util.chaining.scalaUtilChainingOps

val ghOrg = "com-lihaoyi"
val ghRepo = "Ammonite"
Expand Down Expand Up @@ -1018,3 +1023,34 @@ def publishSonatype(
x: _*
)
}

/**
* Somethime, the Mill publish command fails although the Sonatype publishing went through.
* This command checks, whether all artifacts are publshed.
* Run with:
* {{{
* mill checkPublishedArtifacts --artifacts __.publishSelfDependency --version {version} --ttl "1 sec"
* }}}
* See also https://github.com/com-lihaoyi/Ammonite/pull/1453
*/
def checkPublishedArtifacts(artifacts: Tasks[Artifact], version: String, ttl: String = "1 hour") = T.command {
val coords = T.sequence(artifacts.value)()
val next = new AtomicInteger(0)
val fut = coords.map { coord =>
Future {
val dep = s"${coord.group}:${coord.id}:${version}"
println(s"[${next.incrementAndGet()}/${coords.size}] Checking ${dep}")
val res = os.proc("cs", "complete-dep", dep, "-l", ttl)
.call().out.text().trim()
// println(res)
Option.when(!res.contains(version))(dep)
}(ExecutionContext.global)
}
val missing = fut.map(Await.result(_, duration.Duration.Inf)).flatten
val msg = if (missing.size == coords.size) s"All artifacts missing for version ${version}"
else s"Missing ${missing.size} of ${coords.size} published artifacts: ${
missing.mkString("\n- ", "\n- ", "")
}"
if(missing.isEmpty) Result.Success(s"All artifacts published for version ${version}")
else Result.Failure(msg)
}

0 comments on commit 8a463ab

Please sign in to comment.