com.datasalt.pangool.utils.test
Class CollectionInputFormat<K,V>

java.lang.Object
  extended by org.apache.hadoop.mapreduce.InputFormat<K,V>
      extended by com.datasalt.pangool.utils.test.CollectionInputFormat<K,V>
Type Parameters:
K -
V -

public abstract class CollectionInputFormat<K,V>
extends org.apache.hadoop.mapreduce.InputFormat<K,V>

A base input format for use a Collection as input for a MapReduce Job. Useful for unit testing. Example of usage: public static class Input extends CollectionInputFormat { public Input() {} public Collection mapToServe() { ArrayList data = new ArrayList(); data.put(new Duple(new LongWritable(1),new Text("Hola colega"))); data.put(new Duple(new LongWritable(2),new Text("De la vega"))); return data; } } ... job.setInputFormatClass(Input.class); ...


Nested Class Summary
static class CollectionInputFormat.CustomInputSplit
           
static class CollectionInputFormat.Duple<K,V>
           
 
Constructor Summary
CollectionInputFormat()
           
 
Method Summary
 org.apache.hadoop.mapreduce.RecordReader<K,V> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
           
abstract  Collection<Map.Entry<K,V>> dataToServe()
          Override and return the list of Key/Value pairs to be served by this input format.
 List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CollectionInputFormat

public CollectionInputFormat()
Method Detail

dataToServe

public abstract Collection<Map.Entry<K,V>> dataToServe()
Override and return the list of Key/Value pairs to be served by this input format.


getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
                                                       throws IOException,
                                                              InterruptedException
Specified by:
getSplits in class org.apache.hadoop.mapreduce.InputFormat<K,V>
Throws:
IOException
InterruptedException

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<K,V> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                        org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                 throws IOException,
                                                                        InterruptedException
Specified by:
createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<K,V>
Throws:
IOException
InterruptedException


Copyright © –2014 Datasalt Systems S.L.. All rights reserved.