Assignment-2

Assignment-2

Analyzing Aadhar Cards Data

Problem statement:
1. Find out the total number of cards approved by States.
2. Find out the total number of cards rejected by states.
3. Find out the total number of cards approved by cities.
4. Find out the total number of cards rejected by cities.


Data Set Description:
The data set consists of the following fields.
State: This field consists of the state names from all over India
City: This field consists of city names in all states
Approved: This fields consists of the total count of approved cards in numbers
Rejected: This field consists of the total count of rejected cards in numbers


All of you are advised to respect the deadline.
Date of Submission: 20 Apr 2016

Assignment for Pig/HIVE/MR

Assignment

Q 1: Find out frequency of occurrence of average temp. in a month.

Q 2: Find out max. and min. temp. of a month/ a year/ a week.

Q 3: Calculate frequency of temp. less than 10 C.

Q 4: Convert degree C to F.

Q 5: Find the sum of temp. in whole year individually.

Q 6: Find Hottest and coldest year.

 USE FOLLOWING LINK FOR DATA: 

ftp://ftp.ncdc.noaa.gov/pub/data/uscrn/products/daily01/

How to change password of "su" in ubuntu

Program Logic for Patitioner using Hadoop

/*Program for Partitioner 


         public static class MyPartitioner implements Partitioner<Text, IntWritable> {

        @Override
        public int getPartition(Text key, IntWritable value, int numPartitions) {

            String myKey = key.toString().toLowerCase();

            if (myKey.equals("hadoop")) {
                return 0;
            }
            if (myKey.equals("data")) {
                return 1;
            } else {
                return 2;
            }
        }

Concept of SizeCount Program

/* Map Function for SizeCount          

            String sentence=value.toString();
            StringTokenizer token=new StringTokenizer(sentence);
            while(token.hasMoreTokens())
            {
               
            value.set(String.valueOf(token.nextToken().length()));
            output.collect(value, new IntWritable(1));
           
            }

/*Reduce Function for SizeCount

            int sum=0;
            while(values.hasNext())
            {
                sum+=values.next().get();
            }
            output.collect(key, new IntWritable(sum));

WordCount Program Structure

HDFS Commands