Skip to content

Release Howto

Jump to bottom
Ruban Monu edited this page Feb 8, 2016 · 20 revisions

This page describes how to release a new version of elephant-bird.

Merge desired changes to master

All releases are created off the master branch, so merge any changes and push them to master. When making changes to master its typically better to submit a pull request rather than changing directly since that creates an issue to reference the discussion around the change.

Update milestone issues

Each issue merged to master should be assigned to the release milestone.

Update documentation

Update the following files for each release:

  • Readme.md - update documentation for major changes users should be aware of
  • Changes.md - update with specific changes in this release

Clean working directory

Remove any project or release related files

git clean -fdx

Running with vagrant

With the new release script in place the easiest way to do a release is with a vagrant ubuntu/debian box

vagrant init debian/contrib-jessie64
vagrant up --provider virtualbox
rsync -aP -e 'ssh -p 2222' ~/.m2 vagrant@localhost: # Copy up your local maven repo cache, and maven creds to bootstrap the vagrant (password should be vagrant)
rsync -aP -e 'ssh -p 2222' ~/.gnupg vagrant@localhost: # Copy up the gpg keyring
rsync -aP -e 'ssh -p 2222' ~/.gitconfig vagrant@localhost: # Copy up the gitconfig (username/email)
vagrant ssh -- -A # Ssh in with forwarding of agent
  sudo apt-get install -qq openjdk-7-jdk
  export /usr/lib/jvm/java-7-openjdk-amd64
  git clone git@github.com:twitter/elephant-bird.git
  cd elephantbird && ./release.sh -c release -r NEXT_VERSION

Once done:

vagrant destroy # cleans up/deletes the vm

To avoid repeatedly entering the gpg passphrase, see: http://unix.stackexchange.com/questions/46960/how-to-configure-gpg-to-enter-passphrase-only-once-per-session

Ensure tests pass

Make sure all the tests pass before publishing a new release. You need libgplcompression and liblzo2 to run the LZO-related tests.

# (optional) Specify which Java to build with; note "-target 1.6" is used even with newer Java versions
export JAVA_HOME=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/
# (optional) Protocol buffers 2.4.1 is required
export PATH=/usr/local/protobuf-2.4.1/bin:$PATH
mvn clean package -Dtest.library.path=/path/to/hadoop-lzo/build/native/Mac_OS_X-x86_64-64/lib:/usr/local/lzo-2.06/lib

Create the new release artifacts

export LD_LIBRARY_PATH="/path/to/native/lzo/libs"
mvn release:prepare && mvn release:perform

Setting LD_LIBRARY_PATH ensures that lzo tests run. These commands take care of a laundry list of details:

  1. Updates version strings in pom.xml files, dropping -SNAPSHOT suffixes.
  2. Creates commit in local git repo containing changes to pom.xml files.
  3. Creates release tag in local git repo naming the prior commit.
  4. Creates fresh clone of release tag and builds from scratch and runs all unit tests.
  5. Creates all release artifacts, including source and javadoc jars.
  6. Uploads release artifacts to new "staging" repository at Sonatype OSS.
  7. Updates version strings in pom.xml files, bumping versions and adding -SNAPSHOT suffixes.
  8. Creates another commit in local git repo with changes to pom.xml files.

Note that both invocations of mvn will prompt you for your GPG password in order to sign the generated release artifacts.

Close staging repository and test release artifacts

Once artifacts are uploaded to a new staging repository, the repository must be "closed" before it is made publicly accessible. Log in to Sonatype OSS, browse to Staging Repositories and check the box next to the new staging repo, then click the Close button in the toolbar at the top of the list.

Sonatype repository id looks like comtwitter-1013, the artifacts are visible externally at https://oss.sonatype.org/content/repositories/comtwitter-1013/. Test the dependency a with an existing project. E.g. for ivy, you can add 'sonatype-staging' in ivysetttings.xml:

    <ibiblio name="sonatype-staging" m2compatible="true"
          root="https://oss.sonatype.org/content/repositories/comtwitter-1013/"/>

Push commits and release tag to Github

Once you have verified the release artifacts in the staging repository are good, push the commits and release tag generated earlier by invocations of mvn release to Github:

git push twitter master
git push twitter <release-tag>

Release the staging repository

Now "release" the staging repository via Sonatype OSS. The process is very similar to that described in earlier sections; Log in, find the staging repo, select it and click Release.

Promote release artifacts from Sonatype OSS to Maven Central

This should happen automatically after a few hours of releasing the staging repository on Sonatype OSS. If this doesn't happen, get in touch with Sonatype.

Announce the release on the mailing list

Let people know about the release! Ping elephantbird-dev@googlegroups.com with an announcement about the release so people know to upgrade.

Clone this wiki locally