com.datasalt.pangool.pig
Class PangoolStoreFunc

java.lang.Object
  extended by org.apache.pig.StoreFunc
      extended by com.datasalt.pangool.pig.PangoolStoreFunc
All Implemented Interfaces:
org.apache.pig.StoreFuncInterface

public class PangoolStoreFunc
extends org.apache.pig.StoreFunc

A Pig's StoreFunc that can be used to save the result of a Pig flow into a Pangool-friendly format. The Schema is lazily discovered with the first seen Pig Tuple. The type correspondence is:

Any other type is unrecognized and an IOException is thrown.


Constructor Summary
PangoolStoreFunc(String... args)
          First argument is schema name, the others are the column names.
 
Method Summary
 org.apache.hadoop.mapreduce.OutputFormat getOutputFormat()
           
 void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
           
 void putNext(org.apache.pig.data.Tuple pigTuple)
           
 void setStoreLocation(String location, org.apache.hadoop.mapreduce.Job job)
           
 
Methods inherited from class org.apache.pig.StoreFunc
checkSchema, cleanupOnFailure, cleanupOnFailureImpl, relToAbsPathForStoreLocation, setStoreFuncUDFContextSignature, warn
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PangoolStoreFunc

public PangoolStoreFunc(String... args)
First argument is schema name, the others are the column names. We do it this way rather than (String, String...) because Pig doesn't recognize the constructor by reflection otherwise.

Method Detail

getOutputFormat

public org.apache.hadoop.mapreduce.OutputFormat getOutputFormat()
                                                         throws IOException
Specified by:
getOutputFormat in interface org.apache.pig.StoreFuncInterface
Specified by:
getOutputFormat in class org.apache.pig.StoreFunc
Throws:
IOException

prepareToWrite

public void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
                    throws IOException
Specified by:
prepareToWrite in interface org.apache.pig.StoreFuncInterface
Specified by:
prepareToWrite in class org.apache.pig.StoreFunc
Throws:
IOException

putNext

public void putNext(org.apache.pig.data.Tuple pigTuple)
             throws IOException
Specified by:
putNext in interface org.apache.pig.StoreFuncInterface
Specified by:
putNext in class org.apache.pig.StoreFunc
Throws:
IOException

setStoreLocation

public void setStoreLocation(String location,
                             org.apache.hadoop.mapreduce.Job job)
                      throws IOException
Specified by:
setStoreLocation in interface org.apache.pig.StoreFuncInterface
Specified by:
setStoreLocation in class org.apache.pig.StoreFunc
Throws:
IOException


Copyright © –2014 Datasalt Systems S.L.. All rights reserved.