org.kepler.hadoop.actor

Class HDFSLoader

java.lang.Object

ptolemy.kernel.util.NamedObj

ptolemy.kernel.InstantiableNamedObj

ptolemy.kernel.Entity<T>

ptolemy.kernel.ComponentEntity<T>

ptolemy.actor.AtomicActor<TypedIOPort>

ptolemy.actor.TypedAtomicActor

org.kepler.hadoop.actor.HDFSLoader

All Implemented Interfaces:

java.lang.Cloneable, Actor, Executable, FiringsRecordable, Initializable, TypedActor, Changeable, Debuggable, DebugListener, Derivable, Instantiable, ModelErrorHandler, MoMLExportable, Moveable, Nameable

public class HDFSLoader
extends TypedAtomicActor

This class load files/directories to HDFS.

Author:

Jianwu Wang (jianwu@sdsc.edu)

Nested Class Summary

Nested classes/interfaces inherited from class ptolemy.kernel.Entity

Entity.ContainedObjectsIterator

Field Summary

Fields

Modifier and Type	Field and Description
StringParameter	configDir DDP engine configuration directory.
Parameter	confirmOverwrite If false, then overwrite the specified file if it exists without asking.
static java.lang.String	CORE_SITE
static java.lang.String	FS_NAME
PortParameter	sourceLocalPaths The local file path to which to read.
PortParameter	targetHDFSPath The target file path on HDFS to which to write.
TypedIOPort	url Written file URL

Fields inherited from class ptolemy.actor.TypedAtomicActor

_typesValid

Fields inherited from class ptolemy.actor.AtomicActor

_actorFiringListeners, _initializables, _notifyingActorFiring, _stopRequested

Fields inherited from class ptolemy.kernel.util.NamedObj

_changeListeners, _changeLock, _changeRequests, _debugging, _debugListeners, _deferChangeRequests, _elementName, _isPersistent, _verbose, _workspace, ATTRIBUTES, CLASSNAME, COMPLETE, CONTENTS, DEEP, FULLNAME, LINKS

Fields inherited from interface ptolemy.actor.Executable

COMPLETED, NOT_READY, STOP_ITERATING

Constructor Summary

Constructors

Constructor and Description
HDFSLoader(CompositeEntity container, java.lang.String name)

Method Summary

Modifier and Type	Method and Description
java.lang.Object	clone(Workspace workspace) Clone the actor into the specified workspace.
boolean	postfire() Load local files from input port to HDFS.
void	preinitialize() Get default value of HDFS Configure.
void	wrapup() Close the writer if there is one.

Methods inherited from class ptolemy.actor.TypedAtomicActor

_containedTypeConstraints, _customTypeConstraints, _defaultTypeConstraints, _fireAt, _fireAt, attributeTypeChanged, clone, isBackwardTypeInferenceEnabled, newPort, typeConstraintList, typeConstraints

Methods inherited from class ptolemy.actor.AtomicActor

_actorFiring, _actorFiring, _declareDelayDependency, addActorFiringListener, addInitializable, connectionsChanged, createReceivers, declareDelayDependency, fire, getCausalityInterface, getDirector, getExecutiveDirector, getManager, initialize, inputPortList, isFireFunctional, isStrict, iterate, newReceiver, outputPortList, prefire, pruneDependencies, recordFiring, removeActorFiringListener, removeDependency, removeInitializable, setContainer, stop, stopFire, terminate

Methods inherited from class ptolemy.kernel.ComponentEntity

_adjustDeferrals, _checkContainer, _getContainedObject, _propagateExistence, getContainer, instantiate, isAtomic, isOpaque, moveDown, moveToFirst, moveToIndex, moveToLast, moveUp, propagateExistence, setName

Methods inherited from class ptolemy.kernel.Entity

_addPort, _description, _exportMoMLContents, _removePort, _validateSettables, connectedPortList, connectedPorts, containedObjectsIterator, getAttribute, getPort, getPorts, linkedRelationList, linkedRelations, portList, removeAllPorts, setClassDefinition, uniqueName

Methods inherited from class ptolemy.kernel.InstantiableNamedObj

_setParent, exportMoML, getChildren, getElementName, getParent, getPrototypeList, isClassDefinition, isWithinClassDefinition

Methods inherited from class ptolemy.kernel.util.NamedObj

_addAttribute, _adjustOverride, _attachText, _cloneFixAttributeFields, _containedDecorators, _copyChangeRequestList, _debug, _debug, _debug, _debug, _debug, _executeChangeRequests, _getIndentPrefix, _isMoMLSuppressed, _markContentsDerived, _notifyHierarchyListenersAfterChange, _notifyHierarchyListenersBeforeChange, _propagateValue, _removeAttribute, _splitName, _stripNumericSuffix, addChangeListener, addDebugListener, addHierarchyListener, attributeChanged, attributeDeleted, attributeList, attributeList, decorators, deepContains, depthInHierarchy, description, description, event, executeChangeRequests, exportMoML, exportMoML, exportMoML, exportMoML, exportMoMLPlain, getAttribute, getAttributes, getChangeListeners, getClassName, getDecoratorAttribute, getDecoratorAttributes, getDerivedLevel, getDerivedList, getDisplayName, getFullName, getModelErrorHandler, getName, getName, getSource, handleModelError, isDeferringChangeRequests, isOverridden, isPersistent, lazyContainedObjectsIterator, message, notifyOfNameChange, propagateValue, propagateValues, removeAttribute, removeChangeListener, removeDebugListener, removeHierarchyListener, requestChange, setClassName, setDeferringChangeRequests, setDerivedLevel, setDisplayName, setModelErrorHandler, setPersistent, setSource, sortContainedObjects, toplevel, toString, validateSettables, workspace

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ptolemy.actor.Actor

createReceivers, getCausalityInterface, getDirector, getExecutiveDirector, getManager, inputPortList, newReceiver, outputPortList

Methods inherited from interface ptolemy.actor.Executable

fire, isFireFunctional, isStrict, iterate, prefire, stop, stopFire, terminate

Methods inherited from interface ptolemy.actor.Initializable

addInitializable, initialize, removeInitializable

Methods inherited from interface ptolemy.kernel.util.Nameable

description, getContainer, getDisplayName, getFullName, getName, getName, setName

Methods inherited from interface ptolemy.kernel.util.Derivable

getDerivedLevel, getDerivedList, propagateValue

Field Detail

sourceLocalPaths

public PortParameter sourceLocalPaths

The local file path to which to read. This is a string with any form accepted by PortParameter.

See Also:

PortParameter

targetHDFSPath

public PortParameter targetHDFSPath

The target file path on HDFS to which to write. This is a string with any form accepted by PortParameter.

See Also:

PortParameter

confirmOverwrite

public Parameter confirmOverwrite

If false, then overwrite the specified file if it exists without asking. If true (the default), then if the file exists, ask for confirmation before overwriting.

configDir

public StringParameter configDir

DDP engine configuration directory.

url

public TypedIOPort url

Written file URL

CORE_SITE

public static final java.lang.String CORE_SITE

See Also:

Constant Field Values

FS_NAME

public static final java.lang.String FS_NAME

See Also:

Constant Field Values

Constructor Detail

HDFSLoader

public HDFSLoader(CompositeEntity container,
                  java.lang.String name)
           throws NameDuplicationException,
                  IllegalActionException

Throws:

NameDuplicationException

IllegalActionException

Method Detail

clone

public java.lang.Object clone(Workspace workspace)
                       throws java.lang.CloneNotSupportedException

Clone the actor into the specified workspace.

Overrides:

clone in class TypedAtomicActor

Parameters:

workspace - The workspace for the new object.

Returns:

A new actor.

Throws:

java.lang.CloneNotSupportedException - If a derived class contains an attribute that cannot be cloned.

See Also:

NamedObj.exportMoML(Writer, int, String), NamedObj.setDeferringChangeRequests(boolean)

postfire

public boolean postfire()
                 throws IllegalActionException

Load local files from input port to HDFS. If the file does not exist, then create it. If the file already exists, then query the user for overwrite.

Specified by:

postfire in interface Executable

Overrides:

postfire in class AtomicActor<TypedIOPort>

Returns:

True if execution can continue into the next iteration.

Throws:

IllegalActionException - If the file cannot be opened or created, or if the user refuses to overwrite an existing file.

preinitialize

public void preinitialize()
                   throws IllegalActionException

Get default value of HDFS Configure.

Specified by:

preinitialize in interface Initializable

Overrides:

preinitialize in class AtomicActor<TypedIOPort>

Throws:

IllegalActionException - If there is an error reading the alwaysFlush parameter.

wrapup

public void wrapup()
            throws IllegalActionException

Close the writer if there is one.

Specified by:

wrapup in interface Initializable

Overrides:

wrapup in class AtomicActor<TypedIOPort>

Throws:

IllegalActionException - If an IO error occurs.