gr.uoa.di.madgik.workflow.adaptor

Class WorkflowJDLAdaptor

java.lang.Object

gr.uoa.di.madgik.workflow.adaptor.WorkflowJDLAdaptor

All Implemented Interfaces:

IWorkflowAdaptor

public class WorkflowJDLAdaptor
extends Object
implements IWorkflowAdaptor

The Class WorkflowJDLAdaptor constructs an ExecutionPlan based on the description of a job defined in a description using the JDL syntax. This description can be of a single job or it can include a DAG of jobs. The JDL description is parsed using the JDLParser and the adaptor then processed the retrieved ParsedJDLInfo to create the ExecutionPlan

The AdaptorJDLResources provided include all the AttachedJDLResource items that are expected as input from the jobs defined. The output resources of the workflow, retrievable as OutputSandboxJDLResource instances, are constructed from the elements found in all the jobs Output Sandbox.

Depending on the configuration, the adaptor will create the ExecutionPlan that will orchestrate the execution of a DAG of jobs either as a series of SequencePlanElement and FlowPlanElement elements or as a single BagPlanElement. The first case allows for a well defined series of operation but since the creation of such a series of constructs is an exercise on graph topological sorting, which as a problem can provide multiple answers that depending on the nature of the original graph might restrict the parallelization factor of the overall DAG, in cases of complex graphs, this case can damage the parallelization capabilities of n execution plan. The second case is much more dynamic. It allows for execution time decision making of the nodes to be executed. This of course comes as a tradeoff with increased complexity at runtime with respect to the well defined plan, but it can provide the optimal parallelization capabilities.

Staging of input files for the executables is performed at a level of Input Sandbox defined for each job. The resources that are attached to the adaptor are stored in the Storage System and are retrieved in the node that hosts that defines the input sandbox that needs them. The files declared in the Output Sandbox of a job are stored in the Storage System and information on the way to retrieve the output is provided through the OutputSandboxJDLResource which is valid after the completion of the execution.

TODO: - Supported jobs are only those of type Normal - The case of node collocation is not handled correctly because multiple BoundaryPlanElement are created. The node used is still a single one but it is contacted multiple times and data locality is not exploited correctly. - The arguments defined for an executable in the respective JDL attribute, when passed to the ShellPlanElement are split using the space character (' ') as a delimiter. This way no space containing phrase can be passed a single argument - The Retry and Shallow Retry attributes of the JDl are treated equally and are used at the level of ShellPlanElement and not at the level of BoundaryPlanElement - After the execution completion not cleanup in the Storage system is done.

Author:

gpapanikos

Constructor Summary

Constructors

Constructor and Description
WorkflowJDLAdaptor() Instantiates a new workflow jdl adaptor.

Method Summary

Methods

Modifier and Type	Method and Description
void	ConstructEnvironmentHints(String Scope)
void	CreatePlan() Based on the provided resources, a call to this method will do all the needed processing and once completed a full execution plan will be created
gr.uoa.di.madgik.execution.plan.ExecutionPlan	GetCreatedPlan() Retrieves the created plan after the successful invocation of the IWorkflowAdaptor.CreatePlan()
Set<IOutputResource>	GetOutput() Retrieves the set of output resources containing info on the products of the workflow after the plan is executed.
void	SetAdaptorResources(IAdaptorResources Resources) Sets the resources that the adaptor needs to construct the plan.
void	SetExecutionId(String executionId) Sets the execution id which will be associated with the created plan
void	SetJDL(File jdlFile) Sets the jdl file
void	SetJDL(String jdlDescription) Sets the jdl description

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WorkflowJDLAdaptor

public WorkflowJDLAdaptor()

Instantiates a new workflow jdl adaptor.

Method Detail

SetAdaptorResources

public void SetAdaptorResources(IAdaptorResources Resources)
                         throws WorkflowValidationException

Sets the resources that the adaptor needs to construct the plan. These must be of type AdaptorJDLResources.

Specified by:

SetAdaptorResources in interface IWorkflowAdaptor

Parameters:

Resources - the resources needed

Throws:

WorkflowValidationException - Problem with the validity of one of the provided elements

See Also:

IWorkflowAdaptor.SetAdaptorResources(gr.uoa.di.madgik.workflow.adaptor.utils.IAdaptorResources)

SetJDL

public void SetJDL(File jdlFile)

Sets the jdl file

Parameters:

jdlFile - the jdl file

SetJDL

public void SetJDL(String jdlDescription)

Sets the jdl description

Parameters:

jdlDescription - the jdl description

SetExecutionId

public void SetExecutionId(String executionId)

Description copied from interface: IWorkflowAdaptor

Sets the execution id which will be associated with the created plan

Specified by:

SetExecutionId in interface IWorkflowAdaptor

Parameters:

executionId - The execution id

CreatePlan

public void CreatePlan()
                throws WorkflowValidationException,
                       WorkflowSerializationException,
                       WorkflowInternalErrorException,
                       WorkflowEnvironmentException

Description copied from interface: IWorkflowAdaptor

Based on the provided resources, a call to this method will do all the needed processing and once completed a full execution plan will be created

Specified by:

CreatePlan in interface IWorkflowAdaptor

Throws:

WorkflowValidationException - Problem with the validity of one of the provided elements

WorkflowSerializationException - Problem with the serialization of one of the provided elements

WorkflowInternalErrorException - An internal error has occurred

WorkflowEnvironmentException - There was a problem with the environment the Workflow Engine is contained

ConstructEnvironmentHints

public void ConstructEnvironmentHints(String Scope)

GetCreatedPlan

public gr.uoa.di.madgik.execution.plan.ExecutionPlan GetCreatedPlan()

Description copied from interface: IWorkflowAdaptor

Retrieves the created plan after the successful invocation of the IWorkflowAdaptor.CreatePlan()

Specified by:

GetCreatedPlan in interface IWorkflowAdaptor

Returns:

the execution plan

GetOutput

public Set<IOutputResource> GetOutput()

Retrieves the set of output resources containing info on the products of the workflow after the plan is executed. the resources are of type OutputSandboxJDLResource. These resources are determined by the Output Sandbox found in the provided JDL.

Specified by:

GetOutput in interface IWorkflowAdaptor

Returns:

A set of the output resources retrieval information

See Also:

IWorkflowAdaptor.GetOutput()