com.datasalt.pangool.tuplemr.avro
Class AvroInputFormat<T>

java.lang.Object
  extended by org.apache.hadoop.mapreduce.InputFormat<K,V>
      extended by org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.avro.mapred.AvroWrapper<T>,org.apache.hadoop.io.NullWritable>
          extended by com.datasalt.pangool.tuplemr.avro.AvroInputFormat<T>
All Implemented Interfaces:
Serializable

public class AvroInputFormat<T>
extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.avro.mapred.AvroWrapper<T>,org.apache.hadoop.io.NullWritable>
implements Serializable

This is a Pangool's implementation of AvroInputFormat. Instead of being configured via Configuration, its state is defined via instantiation using Java-serialization.

See Also:
Serialized Form

Constructor Summary
AvroInputFormat(org.apache.avro.Schema schema)
           
AvroInputFormat(org.apache.avro.Schema schema, boolean isReflect)
           
 
Method Summary
 org.apache.hadoop.mapreduce.RecordReader<org.apache.avro.mapred.AvroWrapper<T>,org.apache.hadoop.io.NullWritable> createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext context)
           
protected  List<org.apache.hadoop.fs.FileStatus> listStatus(org.apache.hadoop.mapreduce.JobContext job)
           
 
Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
addInputPath, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, isSplitable, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

AvroInputFormat

public AvroInputFormat(org.apache.avro.Schema schema)

AvroInputFormat

public AvroInputFormat(org.apache.avro.Schema schema,
                       boolean isReflect)
Method Detail

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.avro.mapred.AvroWrapper<T>,org.apache.hadoop.io.NullWritable> createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                                                                                                                                            org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                                     throws IOException,
                                                                                                                                            InterruptedException
Specified by:
createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<org.apache.avro.mapred.AvroWrapper<T>,org.apache.hadoop.io.NullWritable>
Throws:
IOException
InterruptedException

listStatus

protected List<org.apache.hadoop.fs.FileStatus> listStatus(org.apache.hadoop.mapreduce.JobContext job)
                                                    throws IOException
Overrides:
listStatus in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<org.apache.avro.mapred.AvroWrapper<T>,org.apache.hadoop.io.NullWritable>
Throws:
IOException


Copyright © –2014 Datasalt Systems S.L.. All rights reserved.