org.kepler.spark.stub

Class KeplerReduceStub

java.lang.Object

org.kepler.spark.stub.PairFlatMapFunctionBase<T,K,V>

org.kepler.spark.stub.KeplerPairFlatMapFunction<java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Iterable<java.lang.Object>>>,java.lang.Object,java.lang.Object>

org.kepler.spark.stub.KeplerReduceStub

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, org.apache.spark.api.java.function.PairFlatMapFunction<java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Iterable<java.lang.Object>>>,java.lang.Object,java.lang.Object>, ExecutionListener

public class KeplerReduceStub
extends KeplerPairFlatMapFunction<java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Iterable<java.lang.Object>>>,java.lang.Object,java.lang.Object>

A Spark stub to run a Kepler workflow in a map after the groupByKey operator.

Author:

Daniel Crawl

See Also:

Serialized Form

Field Summary

Fields inherited from class org.kepler.spark.stub.KeplerPairFlatMapFunction

_keplerManagerException, _manager, _runWorkflowLifecyclePerInput, _sinkActor, _sourceActor

Fields inherited from class org.kepler.spark.stub.PairFlatMapFunctionBase

_parameters

Constructor Summary

Constructors

Constructor and Description
KeplerReduceStub()

Method Summary

Modifier and Type	Method and Description
java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Object>>	call(java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Iterable<java.lang.Object>>> iterator) Transfer the data from Spark to Kepler, execute the workflow, and then transfer the data from Kepler to Spark.

Methods inherited from class org.kepler.spark.stub.KeplerPairFlatMapFunction

_cleanup, _initialize, executionError, executionFinished, managerStateChanged

Methods inherited from class org.kepler.spark.stub.PairFlatMapFunctionBase

readExternal, setConfiguration, writeExternal

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KeplerReduceStub

public KeplerReduceStub()

Method Detail

call

public java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Object>> call(java.util.Iterator<scala.Tuple2<java.lang.Object,java.lang.Iterable<java.lang.Object>>> iterator)
                                                                         throws java.lang.Exception

Transfer the data from Spark to Kepler, execute the workflow, and then transfer the data from Kepler to Spark.

Throws:

java.lang.Exception