org.kepler.spark.operator

Class FileDataSource

java.lang.Object

org.kepler.spark.operator.Operator

org.kepler.spark.operator.DataSource

org.kepler.spark.operator.FileDataSource

public class FileDataSource
extends DataSource

A data sink that reads from files.

Author:

Daniel Crawl

Field Summary

Fields inherited from class org.kepler.spark.operator.DataSource

_context

Fields inherited from class org.kepler.spark.operator.Operator

_configuration, _inputData, _numInstances, _stub

Constructor Summary

Constructors

Constructor and Description
FileDataSource(java.lang.Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> inputFormatClass, java.net.URI path, java.lang.String name) Create a new FileDataSource.

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaPairRDD<java.lang.Object,?>	execute() Execute the operator.

Methods inherited from class org.kepler.spark.operator.DataSource

setContext

Methods inherited from class org.kepler.spark.operator.Operator

_getOutput, getParameters, numInputs, setDegreeOfParallelism, setInput, setInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileDataSource

public FileDataSource(java.lang.Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?>> inputFormatClass,
                      java.net.URI path,
                      java.lang.String name)

Create a new FileDataSource.

Parameters:

inputFormatClass - the format class to read the file

path - the input file path

name - the name of the operator

Method Detail

execute

public org.apache.spark.api.java.JavaPairRDD<java.lang.Object,?> execute()

Execute the operator.

Specified by:

execute in class Operator