Skip to content

Advanced Hera Features

This section is used to publicize Hera’s features beyond the essentials covered in the walk through. Note that these features do not exist in Argo as they are specific to the hera module.

Pre-Build Hooks

Hera offers a pre-build hook feature through hera.shared.register_pre_build_hook with huge flexibility to do pre-build processing on any type of template or Workflow. For example, it can be used to conditionally set the image of a Script, or set which cluster to submit a Workflow to.

To use this feature, you can write a function that takes an object of type template or Workflow, does some processing on the object, then returns it.

For a simple example, we’ll write a function that adds an annotation with key “hera”, value “This workflow was submitted through Hera!”

from hera.shared import register_pre_build_hook
from hera.workflows import Workflow

@register_pre_build_hook
def set_workflow_default_labels(workflow: Workflow) -> Workflow:
    if workflow.annotations is None:
        workflow.annotations = {}

    workflow.annotations["hera-annotation"] = "This workflow was submitted through Hera!"
    return workflow

Now, any time build is called on the Workflow (e.g. to submit it or dump it to yaml), it will add in the annotation!

Load YAML from File

Hera’s Workflow classes offer a collection of to and from functions for dict, yaml and file. This means you can load YAML files and manipulate them as Hera objects!

    with Workflow.from_file("./workflow.yaml") as w:
        w.entrypoint = "my-new-dag-entrypoint"

        with DAG(name="my-new-dag-entrypoint"):
            ...  # Add some tasks!

    w.create()  # And submit to Argo directly from Hera!

The following are all valid assertions:

with Workflow(name="w") as w:
    pass

assert w == Workflow.from_dict(w.to_dict())
assert w == Workflow.from_yaml(w.to_yaml())
assert w == Workflow.from_file(w.to_file())

Submit WorkflowTemplates and ClusterWorkflowTemplates as Workflows

This feature is available for WorkflowTemplates and ClusterWorkflowTemplates, and helps you, as a dev, iterate on your WorkflowTemplate until it’s ready to be deployed. Calling create_as_workflow on a WorkflowTemplate will create a Workflow on the fly which is submitted to the Argo cluster directly and given a generated name, meaning you don’t need to first submit the WorkflowTemplate itself! What this means is you don’t need to keep deleting your WorkflowTemplate and submitting it again, to then run argo submit --from WorkflowTemplate/my-wt while iterating on your WorkflowTemplate.

with WorkflowTemplate(
    name="my-wt",
    namespace="my-namespace",
    workflows_service=ws,
) as wt:
    cowsay = Container(name="cowsay", image="docker/whalesay", command=["cowsay", "foo"])
    with Steps(name="steps"):
        cowsay()

wt.create_as_workflow(generate_name="my-wt-test-1-")  # submitted and given a generated name by Argo like "my-wt-test-1-abcde"
wt.create_as_workflow()  # submitted and given a generated name by Argo like "my-wtabcde"
wt.create_as_workflow()  # submitted and given a generated name by Argo like "my-wtvwxyz"

generate_name is an optional parameter in case you want to control the exact value of the generated name, similarly to the regular Workflow, otherwise the name of the WorkflowTemplate will be used verbatim for generate_name. The Workflow submitted will always use generate_name so that you can call it multiple times in a row without naming conflicts.

Experimental Features

From time to time, Hera will release a new feature under the “experimental feature” flag while we develop the feature and ensure stability. Currently this is used for the RunnerScriptConstructor seen in the runner script example.

To enable experimental features you must set the feature by name to True in the global_config.experimental_features dictionary before using the feature:

global_config.experimental_features["script_runner"] = True

Comments