MapReduce

Hadoop Map-Reduce Design Patterns

How to use the MapReduce:

Clone the repository:

git clone git@github.com:geftimov/MapReduce.git

Go in to the folder:

cd MapReduce

Build it with Maven:

mvn clean install

Example run in each individual pattern example.

Summarization Patterns

#####1. Numerical Summarization ReadMe

• CommentWordCount
• MinMaxCount
• Average
• MedianStdDev (With In-Memory Map)
• MedianAndStandardDeviationCommentLengthByHour (Without the Map, more efficient)

#####2. Inverted Index Summarization ReadMe

• WikipediaExtractor

#####3. Counting with Counters ReadMe

• CountNumUsersByState

Filtering Patterns

#####1. Filtering ReadMe

• DistributedGrep
• SimpleRandomSampling

#####2. Bloom Filtering ReadMe

#####3. Top Ten ReadMe

• TopTenUsersByReputation

#####4. Distinct ReadMe

• DistinctUser

Data Organization Patterns

#####1. Structured to Hierarchical ReadMe

• PostCommentHierarchy
• QuestionAnswerBuilder

#####2. Partitioning ReadMe

• LastAccessDate

#####3. Binning ReadMe

• BinningTags

#####4. TotalOrderSorting ReadMe

• TotalOrderSortingStage

#####5. Shuffling ReadMe

• Anonymize

Join Patterns

#####1. Reduce Side Join ReadMe

• UserJoin
• UserJoinBloomFilter

#####2. Replicated Join ReadMe

• ReplicatedUserJoin

#####3. Composite Join ReadMe

• CompositeUserJoin

#####4. Cartesian Product ReadMe

• CartesianCommentComparison

About Me

Georgi Kalinov Eftimov
jokatavr@gmail.com