Skip to main content

Trigger pipelines on a new artifact

note

Currently, this feature is behind the feature flag CD_TRIGGERS_REFACTOR. Contact Harness Support to enable the feature.

You can trigger Harness pipelines in response to a new artifact version being added to a registry.

For example, every time a new Docker image is pushed to your Docker Hub account, it triggers a CD pipeline that deploys it automatically.

On New Artifact Triggers listen to the registry where one or more of the artifacts in your pipeline are hosted.

You can set conditions on the triggers, such as matching a Docker tag or label or a traditional artifact build name or number.

This trigger is a simple way to automate deployments for new builds.

note

An artifact source does not need to be defined in the service definition for the trigger to work. The only possible scenario of failure is during the initial collection of the artifact within one minute of creating the trigger. For instance, suppose the Docker registry contains 10 tags for a specific image and a trigger is created. In that case, the delegate's polling job retrieves all 10 tags and sends them to the manager, which does not initiate any pipelines. This is because running the pipeline for all 10 tags that were pushed before creation of the trigger could leave the system in an undesirable state. However, when an 11th or any subsequent tag is pushed, the trigger executes and initiates the pipeline.

info

Harness variable expressions are not supported in triggers for pipeline or stage variables.

Supported artifact providers for artifact triggers​

You can use the following artifact providers to triggers pipelines:

  • ACR (Azure Container Registry)
  • Amazon S3
  • Artifactory Registry
  • Bamboo
  • Custom Artifact
  • Docker Registry
  • ECR (Amazon Elastic Container Registry)
  • GCR (Google Container Registry)
  • Github Package Registry
  • Google Artifact Registry
  • Google Cloud Storage
note

The following artifact providers are supported behind the feature flag CD_TRIGGER_V2:

  • Jenkins
  • Azure Artifacts
  • Nexus3 Registry
  • Amazon Machine Image (AMI)

Contact Harness Support to enable the feature.

Important notes​

  • One artifact triggers deployment: If more than one artifact is collected during the polling interval (one minute), only one deployment will be started and will use the last artifact collected.

  • All artifacts trigger deployment: All artifacts collected during the polling interval will trigger a deployment, with one deployment triggered for each artifact collected.

    note

    To enable this feature, go to your Harness project/org/account Default Settings, select Pipeline, and then enable Execute Triggers With All Collected Artifacts or Manifests.

  • Trigger based on file name: The trigger is executed based on file names and not metadata changes.

  • Do not trigger on the latest tag of an artifact, such as a Docker image. With latest, Harness only has metadata, such as the tag name, which has not changed, and so Harness does not know if anything has changed. The trigger will not be executed.

  • In Harness, you can select who is able to create and use triggers within Harness, but you must use your repository's RBAC to control who can add the artifacts or initiate the events that start the Harness trigger.

  • Whenever you create a trigger for the first time, Harness recommends submitting a tag or pushing an artifact to verify its functionality. By doing this, the trigger will execute and the pipeline will run as expected when subsequent tags are pushed.

    note

    When you link a Docker repository to a trigger, the trigger status will remain pending until there are available tags. After the first artifact push, the trigger status changes to success because of new tags, but this alone will not activate the pipeline. The pipeline will only be triggered after a second push to Docker.

  • Whenever a trigger is created or updated, it takes about five to ten minutes for the polling job to start, and for the trigger to be in a working state. Harness recommends that you wait for five to ten minutes after a trigger is created or updated to push the artifact.

  • The polling stops when you disable a trigger. Artifact polling restarts after reenabling the trigger. Harness recommends that you submit a tag or push an artifact and verify the flow as this is treated as a new polling job.

  • Due to a Docker API limitation, image build numbers/tags are always listed in lexical order. To ensure that executions are triggered with the image pushed last, a best practice is to create build numbers or tags that can be sorted lexically using their creation date. Using this method, higher build numbers are assigned for higher creation dates. This ensures that the image pushed last is used when more than one image is pushed over a short period of time, such as less than 5 minutes.

Familiarize yourself with Harness CD pipelines, such as the one you create in the Kubernetes CD Quickstart.

Visual summary​

This 5 minutes video walks you through building an app from source code and pushing it to Docker Hub using Harness CIE, and then having an On New Artifact Trigger execute a CD pipeline to deploy the new app release automatically.

Artifact polling​

Once you have created a trigger to listen for new artifacts, Harness will poll for new artifacts continuously.

Polling is immediate because Harness uses a perpetual task framework that constantly monitors for new builds/tags.

Using the <+trigger.artifact.build> and <+lastPublished.tag> expressions​

When you add a Harness service to the CD stage, you can set the artifact tag to use in Artifacts Details.

If you use a Fixed Value for the artifact Tag (for example, 2), when the trigger executes the pipeline, Harness will deploy the artifact with that tag (2).

If you want the pipeline to deploy the artifact version that initiated the trigger, use the expression <+trigger.artifact.build>.

If you want the pipeline to deploy the last successful published artifact version, use the expression, <+lastPublished.tag>.

note

The lastPublished tag returns the lexicographically last published tag for container image based artifact sources.

last published artifact

If you want the pipeline to deploy the last successful published artifact version of matching regex, use the expression, <+lastPublished.tag>.regex(regex).

last published artifact regex

You can also set tag as a runtime input and then use <+trigger.artifact.build> in the trigger's Pipeline Input settings.

Create an artifact trigger​

  1. Select a Harness pipeline that includes an artifact in the stage's Service Definition.

    You reference an artifact in the stage's service definition in your manifests using the expression <+artifact.image>. Go to Add Container Images as Artifacts for Kubernetes Deployments for more details.

  2. Select Triggers.

  3. Select New Trigger.

  4. The On New Artifact Trigger options are listed under Artifact. Each of the Artifact options are described below.

  5. Select the artifact registry where your artifact is hosted. If you artifact is hosted on Docker Hub and you select GCR, you won't be able to set up your trigger.

  1. In Configuration, in Name, enter a name for the trigger.

  2. In Listen on New Artifact, select Define Artifact Source. This is where you tell Harness what artifact repository to poll for changes.

  3. Create or select the connector to connect Harness to the repository, and then select Continue. For steps on Docker Registry connectors, go to Add Docker Registry Artifact Servers.

  4. In Artifact Details, enter the artifact for this trigger to listen for and select Submit. For example, in Docker Hub, you might enter library/nginx. The artifact is now listed in trigger.

  5. Select Continue.

    In your Docker Registry connector, to connect to a public Docker registry like Docker Hub, use https://registry.hub.docker.com/v2/. To connect to a private Docker registry, use https://index.docker.io/v2/.

Set conditions​

In Conditions, enter any conditions that must be matched in order for the trigger to execute.

Regex and wildcards​

You can use wildcards in the condition's value and you can select Regex.

For example, if the build is todolist-v2.0:

  • With regex not selected, both todolist* or *olist* will match.
  • With regex selected, the regex todolist-v\d.\d will match.

If the regex expression does not result in a match, Harness ignores the value.

Harness supports standard Java regex. For example, if regex is enabled and the intent is to match any branch, the wildcard should be .* instead of simply a wildcard *. If you wanted to match all of the files that end in -DEV.tar you would enter .*-DEV\.tar.

Set metadata conditions​

On New Artifact Triggers support conditions based on artifact metadata expressions.

You can define conditions based on metadata apart from the artifact build and JEXL conditions.

To configure a condition based on artifact metadata, do the following:

  1. In the configuration of an artifact trigger, select Conditions.
  2. In Metadata Conditions, select Add.
  3. Enter an expression in Attribute.
  4. Select an operator and a value to match to the metadata attribute when the expression is resolved.

When the trigger is executed, the metadata condition is evaluated and, if the condition matches, the pipeline is executed.

Here are the artifact metadata expressions you can use:

You can use the following expressions:

<+pipeline.stages.DS.spec.artifacts.primary.metadata.image>
<+pipeline.stages.DS.spec.artifacts.primary.metadata.tag>
<+pipeline.stages.DS.spec.artifacts.primary.metadata.SHAV2>
<+pipeline.stages.DS.spec.artifacts.primary.metadata.SHA>
<+pipeline.stages.DS.spec.artifacts.primary.metadata.url>
<+pipeline.stages.DS.spec.artifacts.primary.dockerConfigJsonSecret>

Define multi-region artifact source​

Once you've selected an artifact in the trigger, you will see the Define Multi Region Artifact Source option.

When artifact repositories such as Google Artifact Registry (GAR) are enabled with multiregion support, artifacts of the same version are available across different regions for easy consumption.

Each region can have similar artifacts. This support enables the configuration of Harness triggers using artifacts from multiple regions.

In On New Artifact triggers, you can configure the regions and conditions associated with the artifact across regions. This enables the pipeline to be triggered based on the availability of artifacts in different regions.

To configure multi region for the artifact, do the following:

  1. In your pipeline, select Triggers.

  2. Create an On New Artifact trigger for your artifact registry.

  3. In Configuration, in the Listen on section, add an artifact. This will be the primary region where the artifact is available. The Harness connector you use should be pointing to the region.

    Once you have added an artifact you will see the Define Multi Region Artifact Source option.

  4. Select Define Multi Region Artifact Source and add artifacts corresponding to other regions. Add as many regions as needed for the trigger.

  5. When you are done, select Continue to move to Conditions.

  6. Select the conditions required for the artifacts across different regions.

    When the artifact version is available across different regions, the condition is evaluated for all the artifacts and the pipeline is triggered.

  7. Complete the trigger setup.

Select pipeline inputs​

If your pipeline uses input sets, you can select the input set to use when the trigger executes the pipeline.

Customize trigger input configuration using override YAML​

To use an input set for both trigger and manual runs,you can easily achieve this by overriding input parameters in trigger inputYAML configuration. This provides the flexibility to modify a specific parameter within the associated Input Set.

For more details, go to Customize trigger input configuration using override YAML

Enable or disable trigger​

You can enable or disable triggers using the enabled toggle:

Reuse trigger YAML to create new triggers​

You can reuse triggers by copying and pasting trigger YAML. This can be helpful when you have advanced conditions you don't want to set up each time.

note

Trigger artifact expressions used in a pipeline are resolved when you rerun a pipeline that was activated by a trigger.

See also​