Hadoop

Thursday, 25 February 2016

Program Logic for Patitioner using Hadoop

/*Program for Partitioner

         public static class MyPartitioner implements Partitioner<Text, IntWritable> {

        @Override
        public int getPartition(Text key, IntWritable value, int numPartitions) {

            String myKey = key.toString().toLowerCase();

            if (myKey.equals("hadoop")) {
                return 0;
            }
            if (myKey.equals("data")) {
                return 1;
            } else {
                return 2;
            }
        }

Concept of SizeCount Program

/* Map Function for SizeCount

String sentence=value.toString();
            StringTokenizer token=new StringTokenizer(sentence);
            while(token.hasMoreTokens())
            {

            value.set(String.valueOf(token.nextToken().length()));
            output.collect(value, new IntWritable(1));

            }

/*Reduce Function for SizeCount

            int sum=0;
           while(values.hasNext())
           {
               sum+=values.next().get();
           }
           output.collect(key, new IntWritable(sum));