org.kepler.spark.operator

Class DataSource

java.lang.Object

org.kepler.spark.operator.Operator

org.kepler.spark.operator.DataSource

Direct Known Subclasses:

FileDataSource, TokenSource

public abstract class DataSource
extends Operator

An operator for reading data.

Author:

Daniel Crawl

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.spark.api.java.JavaSparkContext	_context The Spark context.

Fields inherited from class org.kepler.spark.operator.Operator

_configuration, _inputData, _numInstances, _stub

Constructor Summary

Constructors

Constructor and Description
DataSource(java.lang.Object stub, java.lang.String name) Create a new DataSource.

Method Summary

Modifier and Type	Method and Description
void	setContext(org.apache.spark.api.java.JavaSparkContext context) Set the spark context.

Methods inherited from class org.kepler.spark.operator.Operator

_getOutput, execute, getParameters, numInputs, setDegreeOfParallelism, setInput, setInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_context

protected org.apache.spark.api.java.JavaSparkContext _context

The Spark context.

Constructor Detail

DataSource

public DataSource(java.lang.Object stub,
                  java.lang.String name)

Create a new DataSource.

Parameters:

stub - the stub for reading the data.

name - the name of the operator.

Method Detail

setContext

public void setContext(org.apache.spark.api.java.JavaSparkContext context)

Set the spark context.