Set Up Git Integration
This guide describes the steps a Harness account admin can take to set up the IDP module. Presently, the module needs to be enabled on request. Read more.
Prerequisite: IDP should be provisioned for the given account. The required resources, services, configurations etc. should be in place before starting the harness catalog onboarding flow
- Only users who are Harness Account Admin or assigned with IDP Admin role having the permission as displayed below can perform the onboarding. Here's the detailed documentation on how to assign roles
Getting Started
- User lands on IDP module by navigating from sidebar after logging into their Harness account.
- We strongly recommend users to follow the onboarding guide selecting the Get Started, for a seamless onboarding resulting in a catalog with software components.
Connector Setup
- Connector setup
Limitations
-
Only the following set of connectors are supported
- Harness Code Repository
- GitHub
- GitHub Enterprise
- GitHub App
- GitLab
- Bitbucket
- Bitbucket Cloud
- Bitbucket Server
- Azure Repository
-
Only HTTP mode is supported. SSH connection type is not supported.
-
API access is needed in the connector for the IDP catalog setup.
- Multiple Connectors with different hostname can be used for a single Git Provider at once.
- While setting up connector, both Account & Repo type for URL is supported.
- Connection through harness platform and delegate is supported.
- You can provide the repository URL to verify repository read permission with the given host and credentials before saving the Git integration.
- Harness Code Repository Enabled
- Harness Code Repository Not Enabled
- In case you are already using Harness Code Repository, a default connector with only Read Permission would be available for the code repo set-up under the same account as that of IDP. Note this connector is uneditable and managed by Harness.
- You can as well create a new connector for other git-providers under the +New Integration. If you already have a connector available which you want to use in IDP to save your
catalog-info.yaml
, then select the connector under Select Git Connector, use a URL to validate and save the connector. If you don't have your connector configured already follow the steps mentioned below.
- Select the Git Provider you want to configure from the available options.
- If you already have a connector available which you want to use in IDP to save your
catalog-info.yaml
, then select the connector under Select Git Connector, use a URL to validate and save the connector. If you don't have your connector configured already follow the steps mentioned below.
- Harness Code Repository
- Azure Repository
- Bitbucket
- GitHub
- GitLab
Limitations
At present you can't view and edit source for software components added from Harness Code Repository, this will be available on the next upgrade.
- We don't allow to create a new connector for Harness Code Repository.
- Select Azure Repo icon followed by Create or Select a Connector.
- From the dropdown under Select Azure Repo Connector and either select an already present connector or create +New Connector.
- Connect to Azure Repos. You can also add multiple organizations as different connectors and use them together.
- After the connection test runs, select Finish to save the connector.
- Select Bitbucket icon followed by Create or Select a Connector.
- From the dropdown under Select Bitbucket Connector and either select an already present connector or create +New Connector.
- Configure the Bitbucket connector settings.
Same secret can't be configured for both Password & Personal Access token as backstage makes use of API token to fetch data. Create a secret with token and configure in the Personal Access token field. Reference for creating token
For Bitbucket Cloud the url needs to have src instead of blob. for eg. https://bitbucket.org/org-name/repo-name/src/branch/harness-services/Organization/default.yaml
- After the connection test runs, select Finish to save the connector.
- Select GitHub icon followed by Create or Select a Connector.
- From the dropdown under Select GitHub Connector either select an already present connector or create +New Connector.
- Configure the GitHub connector settings. You can configure connectors for both
github.com
andGitHub Enterprise
and use them at once to fetchcatalog-info.yaml
from both the sources at same time. - After the connection test runs, select Finish to save the connector.
More instructions on using github app in github connector.
- Select GitLab icon followed by Create or Select a Connector.
- From the dropdown under Select GitLab Connector either select an already present connector or create +New Connector.
- Configure the GitLab connector settings. You can configure connectors for both
gitlab.com
andGitLab on-prem
and use them at once to fetchcatalog-info.yaml
from both the sources at same time. - After the connection test runs, select Finish to save the connector.
The scope is tied to URL format https://gitprovider.com/org-name
, so all the Git Providers must provide the URL until at least the org name. Further, it can be scoped down to the repository or project level as well.
- Once the connectors are created, you can see all the git providers configured for IDP.
Onboard Services
- Users will now have option to onboard existing harness services into IDP, or start with a sample service.
- Import Harness Services
- Start with Sample Service
-
User will be shown the list of services in their account. It will be defined in terms of IDP entity i.e harness organization is a domain, harness project is a system and harness service is a component in the IDP world. This list includes services at all scopes.
-
User can choose all the services / individual services / no services
- ALL - All the harness services will be imported into IDP
- INDIVIDUAL - Selected harness services will be imported into IDP
- NO - One sample entity will be imported into IDP for the user to get started.
- User gets a view on how the entity definition looks like
- Now add the details on where the entities will be created in git:
- Harness Code Repository YAML Path
- Other Git Providers YAML Path
-
Connector - The connector is selected by default.
-
Directory Path - Give a path for the directory in which you want to write the
catalog-info.yaml
files.
- Repo Path - Go to the Code Repository and under files select Clone and copy the repository path.
Once you copy the repository path replace the git.eu.harness.io
with accounts.eu.harness.io
.
Validate the permission and the catalog-info.yaml files would be created in your directory in Code repository.
-
Connector - Select the connector of the git provider you want to use.
-
Repo - Enter the full path to your repo. Example
https://github.com/sathish-soundarapandian/onboarding-test.git
https://github.com/sathish-soundarapandian/onboarding-test
- The provided repo in the repo URL should belong to the same organization / project for which the connector has been setup. Ex -
In connector - The account path is https://github.com/sathish-soundarapandian
. So the repo url in IDP onboarding flow should be https://github.com/sathish-soundarapandian/{SOME_REPO}
. It cannot be https://github.com/{SOMETHING_ELSE}/{SOME_REPO}
- this will not work.
-
Provided repo should exist with a valid default HEAD branch. Ideally this will be case when the repo is initialized with README file
-
Branch - Can be new branch / some existing branch. In both the cases, the commit will be done on top of the base HEAD branch.
-
Path - Defaults to harness-services. Can be changed as well.
- User will be shown a demo service metadata in the form of the
catalog-info.yaml
.
- Now this will be added to the git provider:
- Harness Code Repository YAML Path
- Other Git Providers YAML Path
-
Connector - The connector is selected by default.
-
Directory Path - Give a path for the directory in which you want to write the
catalog-info.yaml
files.
- Repo Path - Go to the Code Repository and under files select Clone and copy the repository path.
Once you copy the repository path replace the git.eu.harness.io
with accounts.eu.harness.io
.
Validate the permission and the catalog-info.yaml files would be created in your directory in Code repository.
-
Connector - Select the connector of the git provider you want to use.
-
Repo - Enter the full path to your repo. Example
https://github.com/sathish-soundarapandian/onboarding-test.git
https://github.com/sathish-soundarapandian/onboarding-test
- The provided repo in the repo URL should belong to the same organization / project for which the connector has been setup. Ex -
In connector - The account path is https://github.com/sathish-soundarapandian
. So the repo url in IDP onboarding flow should be https://github.com/sathish-soundarapandian/{SOME_REPO}
. It cannot be https://github.com/{SOMETHING_ELSE}/{SOME_REPO}
- this will not work.
-
Provided repo should exist with a valid default HEAD branch. Ideally this will be case when the repo is initialized with README file
-
Branch - Can be new branch / some existing branch. In both the cases, the commit will be done on top of the base HEAD branch.
-
Path - Defaults to harness-services. Can be changed as well.
Create and Register Entities
-
Once the required details are entered and submitted for importing, we will push the generated entity yaml files to the repo and path provided. You will be seeing two commits - One is done during the sync flow and other commit which will consist of remaining entities will be pushed in an async manner. Time frame for async operation will depend on how big the repo size is and how many revisions the provided repo has.
-
In the background, the catalog are also imported into IDP along with their associated configs.
Catalog
-
User can navigate to the IDP homepage to get started. Catalog will be start showing up once the async operation is completed.
-
Since during the onboarding flow harness will not be able to discover the complete metadata of the entity, the owner field will be set to Unknown. As part of the IDP provisioning, harness users and groups are imported to IDP. With this in place, customers can start editing the catalog info yaml definition to update the owner for each of the entity.
- Post onboarding, users can import any number of entities into harness-idp using the register component flow. This flow expects you to provide the complete url where the entity definition is hosted.
Recommendations
If using GitHub connector, customers can go with App based authentication which provides higher number of API requests in an hour window. Read more about GitHub Apps - GitHub Docs.
Next steps
Check out the IDP tutorials for walkthroughs and deep dives into specific IDP use cases.