Skip to content
forked from etsy/jading

Tool for packaging cascading.jruby scripts for execution via hadoop jar

License

Notifications You must be signed in to change notification settings

mrwalker/jading

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

30 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Jading

Jading is a build and execution tool for cascading.jruby that handles packaging all the dependencies of your scripts into a jar for execution on a Hadoop cluster.

The primary entry point to Jading is the script "jade." This script operates in two modes, build and execution. The first allows you to produce jade jars, which contain everything you need to run a cascading.jruby job on a Hadoop cluster. The second is a convenience wrapper for hadoop jar that selects the correct "runner" code from Jading's library to run your job remotely.

Example Usage

Running:

jade \
  -g jruby-openssl \
  -g json \
  -l lib/operators/target/custom-cascading-operators-0.0.2.jar \
  -l lib/operators/target/emr \
  lib/jobs/helpers \
  lib/ruby/cli.rb \
  lib/ruby/barnum_date.rb \
  lib/jobs/cascading/nightly/index_events.rb

Produces jade.jar in the current directory, which can be executed on a Hadoop cluster using:

jade -e index_events.rb yesterday_path=2012_04_26 input_prefix=hdfs://logs.etsy.com ...

For more details, see jade -h and Getting Started in the wiki.

About

Tool for packaging cascading.jruby scripts for execution via hadoop jar

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 63.7%
  • Ruby 36.3%