| title | titleSuffix | ms.custom | description | ms.topic | ms.assetid | ms.author | ms.date | monikerRange |
|---|---|---|---|---|---|---|---|---|
Configure and pay for parallel jobs |
Azure DevOps |
seodec18,contperf-fy21q1 |
Configure parallel jobs in Azure Pipelines and pay for them |
how-to |
FAFB2DE4-F462-4E9E-8312-4F343F2A35B8 |
jukullam |
02/25/2022 |
<= azure-devops |
[!INCLUDE version-lt-eq-azure-devops]
::: moniker range="< azure-devops-2019"
This article describes the licensing model for Azure Pipelines in Team Foundation Server 2017 (TFS 2017) or newer. We don't charge you for Team Foundation Build (TFBuild) so long as you have a TFS Client Access License (CAL).
A TFS parallel job gives you the ability to run a single release at a time in a project collection. You can keep hundreds or even thousands of release jobs in your collection. But, to run more than one release at a time, you need additional parallel jobs.
One free parallel job is included with every collection in a Team Foundation Server. Every Visual Studio Enterprise subscriber in a Team Foundation Server contributes one additional parallel job.
You can buy additional private jobs from the Visual Studio Marketplace. There is a maximum limit of 25 parallel jobs for Microsoft-hosted agents.
::: moniker-end
::: moniker range=">= azure-devops-2019 < azure-devops"
Important
Starting with Azure DevOps Server 2019, you do not have to pay for self-hosted concurrent jobs in releases. You are only limited by the number of agents that you have.
::: moniker-end
::: moniker range="azure-devops"
Learn how to estimate how many parallel jobs you need and buy more parallel jobs for your organization.
Note
We have temporarily disabled the free grant of parallel jobs for public projects and for certain private projects in new organizations. However, you can request this grant by submitting a request. Existing organizations and projects are not affected. Please note that it takes us 2-3 business days to respond to your free tier requests.
When you define a pipeline, you can define it as a collection of jobs. When a pipeline runs, you can run multiple jobs as part of that pipeline. Each running job consumes a parallel job that runs on an agent. When there aren't enough parallel jobs available for your organization, the jobs are queued up and run one after the other.
In Azure Pipelines, you can run parallel jobs on Microsoft-hosted infrastructure or your own (self-hosted) infrastructure. Each parallel job allows you to run a single job at a time in your organization. You do not need to pay for parallel jobs if you are using an on-premises server. The concept of parallel jobs only applies to Azure DevOps Services.
If you want to run your jobs on machines that Microsoft manages, use Microsoft-hosted parallel jobs. Your jobs will run on Microsoft-hosted agents.
If you want Azure Pipelines to orchestrate your builds and releases, but use your own machines to run them, use self-hosted parallel jobs. For self-hosted parallel jobs, you'll start by deploying our self-hosted agents on your machines. You can register any number of these self-hosted agents in your organization.
::: moniker-end
::: moniker range="azure-devops"
We provide a free tier of service by default in every organization for both hosted and self-hosted parallel jobs. Parallel jobs are purchased at the organization level, and they are shared by all projects in an organization.
For Microsoft-hosted parallel jobs, you can get up to 10 free Microsoft-hosted parallel jobs that can run for up to 360 minutes (6 hours) each time for public projects. When you create a new Azure DevOps organization, you are not given this free grant by default.
For private projects, you can get one free job that can run for up to 60 minutes each time. When you create a new Azure DevOps organization you may not always be given this free grant by default.
To request the free grant for public or private projects, submit a request.
Note
It takes us 2-3 business days to respond to your free tier request.
There is no time limit on parallel jobs for public projects and a 30 hour time limit per month for private projects.
| Number of parallel jobs | Time limit | |
|---|---|---|
| Public project | Up to 10 free Microsoft-hosted parallel jobs that can run for up to 360 minutes (6 hours) each time | No overall time limit per month |
| Private project | One free job that can run for up to 60 minutes each time | 1,800 minutes (30 hours) per month |
When the free tier is no longer sufficient, you can pay for additional capacity per parallel job. For pricing cost per parallel job, see the Azure DevOps pricing page. Paid parallel jobs remove the monthly time limit and allow you to run each job for up to 360 minutes (6 hours).
Buy Microsoft-hosted parallel jobs.
New organizations have a maximum limit of 25 parallel jobs for Microsoft-hosted agents. Contact support to request a limit increase, subject to capacity in your organization's region.
When you purchase your first Microsoft-hosted parallel job, the number of parallel jobs you have in the organization is still one. To be able to run two jobs concurrently, you will need to purchase two parallel jobs if you are currently on the free tier. The first purchase only removes the time limits on the first job.
Tip
If your pipeline exceeds the maximum job timeout, try splitting your pipeline into multiple jobs. For more information on jobs, see Specify jobs in your pipeline.
For self-hosted parallel jobs, you can register any number of self-hosted agents in your organization. We charge based on the number of jobs you want to run at a time, not the number of agents registered. There are no time limits on self-hosted jobs.
For public projects that are self-hosted, you can have unlimited parallel jobs running. For private projects, you can have one job and one additional job for each active Visual Studio Enterprise subscriber who is a member of your organization.
| Number of parallel jobs | Time limit | |
|---|---|---|
| Public project | Unlimited | None |
| Private project | One self-hosted job; For each active Visual Studio Enterprise subscriber who is a member of your organization, you get one additional self-hosted parallel job. | None |
When the free tier is no longer sufficient for your self-hosted private project, you can purchase more additional capacity per parallel job. For pricing cost per parallel job, see the Azure DevOps pricing page.
Buy self-hosted parallel jobs.
::: moniker-end
::: moniker range="azure-devops"
As the number of queued builds and releases exceeds the number of parallel jobs you have, your build and release queues will grow longer. When you find the queue delays are too long, you can purchase additional parallel jobs as needed. There are several methods you can use to check your parallel job limits and job history.
You can use the Pool consumption report, available on the Analytics tab of your agent pool, to see a chart of running and queued jobs graphed with your parallel jobs for the previous 30 days. If you have a backlog of queued jobs and your running jobs are at the concurrency limit, you may wish to purchase more parallel jobs. For more information, see Pool consumption report.
:::image type="content" source="../agents/media/pool-consumption-report/historical-graph-azure-pipelines.png" alt-text="Microsoft-hosted agent pool historical graph":::
Figure out how many parallel jobs you need by first seeing how many parallel jobs your organization currently uses:
-
Browse to Organization settings > Pipelines > Retention and parallel jobs > Parallel jobs.
:::image type="content" source="media/concurrent-pipelines-vsts/control-panel-account-build-and-release-resource-limits.png" alt-text="Location of parallel jobs in organization settings.":::
URL example:
https://{your_organization}/_admin/_buildQueue?_a=resourceLimits -
View the maximum number of parallel jobs that are available in your organization.
-
Select View in-progress jobs to display all the builds and releases that are actively consuming an available parallel job or that are queued waiting for a parallel job to be available.
A simple rule of thumb: Estimate that you'll need one parallel job for every four to five users in your organization.
In the following scenarios, you might need multiple parallel jobs:
- If you have multiple teams, and if each of them require CI, you'll likely need a parallel job for each team.
- If your CI trigger applies to multiple branches, you'll likely need a parallel job for each active branch.
- If you develop multiple applications by using one organization or server, you'll likely need additional parallel jobs: one to deploy each application at the same time.
::: moniker-end
::: moniker range="azure-devops"
To buy more parallel jobs:
- Billing must be set up for your organization
- You need to be a member of the Project Collection Administrators group.
Buy more parallel jobs within your organization settings:
-
Sign in to your organization (
https://dev.azure.com/{yourorganization}). -
Select
Organization settings.
-
Select Parallel jobs under Pipelines, and then select either Purchase parallel jobs for Microsoft-hosted jobs or Change for self-hosted jobs.
:::image type="content" source="../../organizations/billing/media/shared/manage-parallel-jobs.png" alt-text="manage parallel jobs image":::
-
Enter your desired amount, and then Save.
For pricing cost per parallel job, see the Azure DevOps pricing page.
-
Sign in to your organization (
https://dev.azure.com/{yourorganization}). -
Select
Organization settings. -
Select Parallel jobs under Pipelines, and then select either Purchase parallel jobs or Change for Microsoft-hosted jobs or Change for self-hosted jobs.
:::image type="content" source="../../organizations/billing/media/shared/manage-parallel-jobs.png" alt-text="image of manage parallel jobs":::
-
Enter a lesser or greater quantity of Microsoft-hosted or self-hosted jobs, and then select Save.
Important
Hosted XAML build controller isn't supported. If you have an organization where you need to run XAML builds, set up an on-premises build server and switch to an on-premises build controller. For more information about the hosted XAML model, see Get started with XAML.
::: moniker-end
::: moniker range="azure-devops"
Consider an organization that has only one Microsoft-hosted parallel job. This job allows users in that organization to collectively run only one job at a time. When additional jobs are triggered, they are queued and will wait for the previous job to finish.
If you use release or YAML pipelines, then a run consumes a parallel job only when it's being actively deployed to a stage. While the release is waiting for an approval or a manual intervention, it does not consume a parallel job.
When you run a server job or deploy to a deployment group using release pipelines, you don't consume any parallel jobs.
- FabrikamFiber CI Build 102 (main branch) starts first.
- Deployment of FabrikamFiber Release 11 is triggered by completion of FabrikamFiber CI Build 102.
- FabrikamFiber CI Build 101 (feature branch) is triggered. The build can't start yet because Release 11's deployment is active. So the build stays queued.
- Release 11 waits for approvals. Fabrikam CI Build 101 starts because a release that's waiting for approvals does not consume a parallel job.
- Release 11 is approved. It resumes only after Fabrikam CI Build 101 is completed.
::: moniker-end
::: moniker range="< azure-devops-2019"
For example, a collection in a Team Foundation Server has one parallel job. This allows users in that collection to run only one release at a time. When additional releases are triggered, they are queued and will wait for the previous one to complete.
A release requires a parallel job only when it is being actively deployed to a stage. Waiting for an approval does not consume a parallel job. However, waiting for a manual intervention in the middle of a deployment does consume a parallel job.
- FabrikamFiber Release 10 is first to be deployed.
- Deployment of FabrikamFiber Release 11 starts after Release 10's deployment is complete.
- Release 12 is queued until Release 11's deployment is active.
- Release 11 waits for an approval. Release 12's deployment starts because a release waiting for approvals does not consume a parallel job.
- Even though Release 11 is approved, it resumes only after Release 12's deployment is completed.
- Release 11 is waiting for manual intervention. Release 13 cannot start because the manual intervention state consumes a parallel job.
Manual intervention does not consume a job in TFS 2017.1 and newer.
Parallel processing within a single release does not require additional parallel jobs. So long as you have enough agents, you can deploy to multiple stages in a release at the same time.
For example, suppose your collection has three parallel jobs. You can have more than three agents running at the same time to perform parallel operations within releases. For instance, notice below that four or five agents are actively running jobs from three parallel jobs.
For example, here's an organization that has multiple Team Foundation Servers. Two of their users have Visual Studio Enterprise subscriptions that they can use at the same time across all their on-premises servers and in each collection so long as the customer adds them as users to both the servers as explained below.
You can begin by seeing if your teams can get by with the parallel jobs you've got by default. As the number of queued releases exceeds the number of parallel jobs you have, your release queues will grow longer. When you find the queue delays are too long, you can purchase additional parallel jobs as needed.
A simple rule of thumb: Estimate that you'll need one parallel job for every 10 users in your server.
In the following scenarios you might need multiple parallel jobs:
-
If you have multiple teams, if each of them require a CI build, and if each of the CI builds is configured to trigger a release, then you'll likely need a parallel job for each team.
-
If you develop multiple applications in one collection, then you'll likely need additional parallel jobs: one to deploy each application at the same time.
Users who have Visual Studio Enterprise subscriptions are assigned to VS Enterprise access level in the Users hub of TFS instance. Each of these users contributes one additional parallel job to each collection. You can use this benefit on all Team Foundation Servers in your organization.
-
Browse to Server settings, Access levels.
URL example:
http://{your_server}:8080/tfs/_admin/_licenses -
On the left side of the page, click VS Enterprise.
-
Add your users who have Visual Studio Enterprise subscriptions.
After you've added these users, additional licenses will appear on the resource limits page described below.
If you need to run more parallel releases, you can buy additional private jobs from the Visual Studio marketplace. Since there is no way to directly purchase parallel jobs from Marketplace for a TFS instance at present, you must first buy parallel jobs for an Azure DevOps organization. After you buy the private jobs for an Azure DevOps organization, you enter the number of purchased parallel jobs manually on the resource limits page described below.
-
Browse to Collection settings, Pipelines, Resource limits.
URL example:
http://{your_server}:8080/tfs/DefaultCollection/_admin/_buildQueue?_a=resourceLimits -
View or edit the number of purchased parallel jobs.
::: moniker-end
::: moniker range="< azure-devops-2019 || azure-devops"
::: moniker-end
::: moniker range="azure-devops"
You qualify for the free tier limits for public projects if you meet both of these conditions:
- Your pipeline is part of an Azure Pipelines public project.
- Your pipeline builds a public repository from GitHub or from the same public project in your Azure DevOps organization.
For information on how to apply for the grant of free parallel jobs, see How much do parallel jobs cost (Microsoft-hosted)?
Currently, there isn't a way to partition or dedicate parallel job capacity to a specific project or agent pool. For example:
- You purchase two parallel jobs in your organization.
- You start two runs in the first project, and both the parallel jobs are consumed.
- You start a run in the second project. That run won't start until one of the runs in your first project is completed.
You can have as many users as you want when you're using Azure Pipelines. There is no per-user charge for using Azure Pipelines. Users with both basic and stakeholder access can author as many builds and releases as they want.
No. You can create hundreds or even thousands of pipelines for no charge. You can register any number of self-hosted agents for no charge.
As a Visual Studio Enterprise subscriber, do I get additional parallel jobs for TFS and Azure Pipelines?
Yes. Visual Studio Enterprise subscribers get one parallel job in Team Foundation Server 2017 or later and one self-hosted parallel job in each Azure DevOps Services organization where they are a member.
Some of our earlier customers are still on a per-minute plan for the hosted agents. In this plan, you pay $0.05/minute for the first 20 hours after the free tier, and $0.01/minute after 20 hours. Because of the following limitations in this plan, you might want to consider moving to the parallel jobs model:
- When you're using the per-minute plan, you can run only one job at a time.
- If you run builds for more than 14 paid hours in a month, the per-minute plan might be less cost-effective than the parallel jobs model.
You can register one XAML build controller for each self-hosted parallel job in your organization. Your organization gets at least one free self-hosted parallel job, so you can register one XAML build controller for no additional charge. For each additional XAML build controller, you'll need an additional self-hosted parallel job.
::: moniker-end
::: moniker range="< azure-devops-2019"
TFS users with a TFS CAL can author as many releases as they want.
To approve releases, a TFS CAL is not necessary; any user with stakeholder access can approve or reject releases.
No, on TFS you don't need parallel jobs to run builds. You can run as many builds as you want at the same time for no additional charge.
No.
In TFS 2015, so long as your users have a TFS CAL, they can manage releases for no additional charge in trial mode. We called it "trial mode" to indicate that we would eventually charge for managing releases. Despite this label, we fully support managing releases in TFS 2015.
::: moniker-end