org.kepler.spark.director

Class SparkJob

java.lang.Object

org.kepler.spark.director.SparkJob

public class SparkJob
extends java.lang.Object

A Spark job that can be cancelled.

Author:

Daniel Crawl

Constructor Summary

Constructors

Constructor and Description
SparkJob(org.apache.spark.api.java.JavaSparkContext context, DataSink operator, java.lang.String name) Create a new job.

Method Summary

Modifier and Type	Method and Description
void	addDataSink(DataSink operator) Add a sink operator to the job.
void	start() Start the job.
void	stop()
void	waitForFinish() Wait until the job in complete.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkJob

public SparkJob(org.apache.spark.api.java.JavaSparkContext context,
                DataSink operator,
                java.lang.String name)

Create a new job.

Parameters:

operator - sink operator

name - job name

Method Detail

start

public void start()

Start the job.

stop

public void stop()

waitForFinish

public void waitForFinish()
                   throws java.lang.Exception

Wait until the job in complete.

Throws:

java.lang.Exception

addDataSink

public void addDataSink(DataSink operator)

Add a sink operator to the job.