Geek asks and answers

MapReduce - how sorting reduces output by value

How can I sort in decreasing order of the output of the gearbox by value? I am developing an application that should return the listened songs. Thus, songs should be ordered by the number of plays. My application works as follows:

Input: songname@userid@boolean
MapOutput : songname userid
ReduceOutput : songname number_of_listening

Any idea how to do this?

+5
source share
2 answers

Per documents , gearbox output not re-sorted. Either collect the gearbox input (if that works for your application) by setting the appropriate value for JobConf.setOutputValueGroupingComparator (class) or just sort the gearbox final output in a separate step.

+2

- MapReduce , Sort.java. Hadoop Map , . - :

public static class Map extends Mapper<LongWritable,Text,IntWritable,Text>{
   private Text word = new Text();
   public void map(LongWritable key, Text value, Context context) throws IO Exception, Interrupted Exception{
   String line = value.toString();
   StringTokenizer tokenizer = new StringTokenizer(line);
   word.set(tokenizer.nextToken());
   IntWritable number = new IntWritable(Integer.parseInt(tokenizer.nextToken()));
   context.write(number,word);
   }     
}

[LongWritable, text] MapReduce LongWritable. , !

CL

+5

More articles:


All Articles