HeartBeat is a tool for tracking project delivery metrics that can help you get a better understanding of delivery performance. This product allows you easily get all aspects of source data faster and more accurate to analyze team delivery performance which enables delivery teams and team leaders focusing on driving continuous improvement and enhancing team productivity and efficiency.
State of DevOps Report is launching in 2019. In this webinar, The 4 key metrics research team and Google Cloud share key metrics to measure DevOps performance, measure the effectiveness of development and delivery practices. They searching about six years, developed four metrics that provide a high-level systems view of software delivery and performance.
Here are the for Key meterics:
- Deployment Frequency (DF)
- Lead Time for changes (LTC)
- Mean Time To Recover (MTTR)
- Change Failure Rate (CFR)
In HeartBeat tool, we also have some other metrics, like: Velocity, Cycle Time and Classification. So we can collect DF, LTC, CFR, Velocity, Cycle Time and Classification.
Here is the user manaul for Version 1 on 2020/06. For now, we just can support Jira/Buildkite/Github to generate the corresponding metrics data.
| Type | board | pipeline | Repo |
|---|---|---|---|
| Support tools | Jira √ Trello × |
Buildkite √ Teamcity × GoCD × |
Github √ Bitbucket × |
Note： “√” means can support, “×” means can not support
Before generator the metrics data, user need to config the project info, in Home page (Image3-1), you can create a new project for your project, or you can import a project config json file (If you already saved one config file, for import file feature will introduce in “Import and Export feature ”).
Image 3-1，home page
If you are first use the product, you need to select “Create A New Project”，it will go to config page (Image 3-2)
Image 3-2，Project config page
Users need to select a period of time, then all of the data that follows is based on that time period.
Have two items of time period:
- Regular Calendar(Weekend Considered): If you select this item, it means all data will exclude the weekend.
- Calendar with Chinese Holiday: If you select this item, it means all data will exclude the weekend and Chinese holiday. So if the time period you selected contains Chinese holiday, you need to select this item.
All need to select which data you want to get, for now, we support six metrics data (Image 3-3).
Image 3-3，Metrics Data
Because all metrics data from different tools that your projects use. Need to have the access to these tools then you can get the data. So after select time period and metrics data, then you need to input the config for different tools(Image 3-4).
Image 3-4，Project config
The details for board:
| Items | Description |
|---|---|
| Board Type | Support two types of board: Classic Jira and Next-gen Jira |
| Board Id | The value of BoardId is number. You need to find it from your team’s Jira board URL. For Example: 1. Your jira board URL like below, then 2 is the boardId https://dorametrics.atlassian.net/jira/software/projects/ADM/boards/2 2. Your jira board URL like below, then rapidView=3, 3 is the boardId https://pokergame.atlassian.net/secure/RapidBoard.jspa?projectKey=KAN1&useStoredSettings=true&rapidView=3 |
| Project | Project is the project key for your project. Also can find it from your team’s Jira board URL. For Example: 1. Your jira board URL like below, then ADM is the project https://dorametrics.atlassian.net/jira/software/projects/ADM/boards/2 2. Your jira board URL like below, then projectKey=KAN1, KAN1 is the project https://pokergame.atlassian.net/secure/RapidBoard.jspa?projectKey=KAN1&useStoredSettings=true&rapidView=3 |
| Site | Site is the domain for your jira board, like below URL, dorametrics is the site https://dorametrics.atlassian.net/jira/software/projects/ADM/boards/2 |
| The email can access to the Jira board | |
| Token | Use the token for the email you used, https://id.atlassian.com/manage-profile/security/api-tokens |
The details for Pipeline:
| Items | Description |
|---|---|
| PipelineTool | The pipeline tool you team use, but just support BuildKit in Version 1 |
| Token | The token can access to pipeline tool, https://buildkite.com/user/api-access-tokens |
The details for SourceControl:
| Items | Description |
|---|---|
| SourceControl | The source control tool you team use, but just support GitHub in Version 1 |
| Token | The token can access to source control tool, https://github.com/settings/tokens |
After inputting the details info, users need to click the “Verify” button to verify if can access to these tool. Once can access, can click the “Next” button go to next page -- Config Metrics page(Image 3-5，Image 3-6，Image 3-7)
Image 3-5, Crews/Cycle Time config
Crew Settings: In the last page, it will get all the tickets that finished in the time period selected in the last step. So also get the all assignees list that assigned for these done tickets. In the crew setting, will list all assignees. Users can select any assignees or all assignees to generate the report.
Cycle Time: It will list all columns for the current active jira board. Then users need to map the each column to the supported columns. Like, if your board have “in progress” column, it means developer doing this ticket, so it should be mapping with “In Dev” for the list we provide.
| Status | Description |
|---|---|
| To do | It means the ticket needs to be done, waiting for Dev to pick it. Cycle time doesn't include this time. |
| Analysis | BA or other people still need to analyze the ticket. Cycle time doesn't include this time. |
| In Dev | It means dev is doing the ticket. This time should be a part of cycle time. And it is named development time. |
| Block | It means the tickets blocked by some issues, cannot be done now. This time should be a part of cycle time. And it is named block time. |
| Waiting for testing | It means waiting for Dev to pick or QA to testing. This time should be a part of cycle time. And it is named waiting time. |
| Testing | It means QA is testing the tickets. This time should be a part of cycle time. And it is named testing time. |
| Review | It means PO or other people are reviewing the tickets. This time should be a part of cycle time. And it is named review time. |
| Done | It means the tickets are already done. Cycle time doesn't include this time. |
| -- | If you don't need to map, you can select -- |
Image 3-6，Classification Settings
In classification settings, it will list all Context fields for your jira board. Users can select anyone to get the data for them.
Image 3-7，Settings for Pipeline
| Items | Description |
|---|---|
| Organization | The organization for your pipelines |
| Pipeline Name | Your pipeline name |
| Steps | Your pipeline steps name |
When user first use this tool, need to create a project, and do some config. To avoid the user entering configuration information repeatedly every time, we provide a “Save” button in the config and metrics pages. In config page, click the save button, it will save all items in config page in a Json file. If you click the save button in the metrics page, it will save all items in config and metrics settings in a Json file. Here is the json file (Image 3-8)。Note: Below screenshot just contains a part of data.
Image 3-8, Config Json file
When user already saved config file before, then you don’t need to create a new project. In the home page, can click Import Project from File button(Image 3-1) to select the config file. If your config file is too old, and the tool already have some new feature change, then if you import the config file, it will get some warning info(Image 3-9). You need to re-select some info, then go to the next page.
Image 3-9, Warning message
After config, then it will generate the report for you.
In Velocity Report, it will list the corresponding data by Story Point and the number of story tickets. (image 3-10)
Image 3-10，Velocity Report
The calculation process data and final result of Cycle Time are calculated by rounding method, and two digits are kept after the decimal point. Such as: 3.567... Is 3.56; 3.564... Is 3.56.
Image 3-11，Cycle Time Report
Image 3-12，Classification Report
Image 3-13，Deployment Frequency Report
Image 3-14，Lead time for changes Report
Image 3-15，Change Failure Rate Report
After generating the report, you can export the original data for your board and pipeline (Image 3-15). Users can click the “Export board data” or “Export pipeline data” button to export the original data.
It will export a csv file for board data. It contains two parts: Part 1: Export the all done tickets during the time period Part 2: Export the all non-done tickets in your current active board. And it will order by ticket status (Image 3-16)
Image 3-16，Exported Board Data
All columns for Jira board:
| Column name | Description |
|---|---|
| Issue key | Ticket ID |
| Summary | -- |
| Issue Type | -- |
| Status | -- |
| Story Point | -- |
| Assignee | -- |
| Reporter | -- |
| Project Key | -- |
| Project Name | -- |
| Priority | -- |
| Parent Summary | The epic for ticket |
| Sprint | Which sprint this ticket in |
| Labels | -- |
| Cycle Time | total Cycle Time |
| Cycle Time / Story Points | Cycle Time for each point |
| Analysis Days | Analysis days for each ticket |
| In Dev Days | Development days for each ticket |
| Waiting Days | After development, how long will take before testing |
| Testing Days | Testing days for each ticket |
| Block Days | Blocked days for each ticket |
| Review Days | -- |
| Original Cycle Time: {Column Name} | The data for Jira board original data |
It will export a csv file for pipeline data (image 3-17).
Image 3-17，Exported Pipeline Data
All columns for pipeline data:
| Column name | Description |
|---|---|
| Pipeline Name | -- |
| Pipeline Step | Step name |
| Committer | -- |
| Code Committed Time | Committed time |
| PR Created Time | -- |
| PR Merged Time | -- |
| Deployment Completed Time | When it finished deploy |
| Total Lead Time (mins) | -- |
| Time from PR Created to PR Merged (mins) | -- |
| Time from PR Merged to Deployment Completed (mins) | -- |
| Status | Status for pipeline (Pass or Failed) |
In the current version, if you add or delete some columns for the jira board, it will change finish time for all last column tickets to add/delete column time. (It just impact Next-gen Jira), here are the details info:
| Jira Template | Add column | Delete column |
|---|---|---|
| Kanban | It will change finish time for all last column tickets to add/delete column time | If delete non-last column: It will change finish time for all last column tickets to add/delete column time If delete the last column: It will change finish time for current last column tickets to add/delete column time |
| Scrum | finish time for all last column tickets to add/delete column time All finished ticket’s finish time changed to add/delete column time |
If delete the last column: It will change finish time for current last column tickets to add/delete column time |
For now, we don’t have a good solution to resolve this issue.
For Classic Jira users, before you use this tool, you need to do some settings for the jira board. Otherwise, you cannot get the data. Here are the steps you need to do:
-
Open https://{site}.atlassian.net/secure/admin/ViewIssueFields.jspa?start=0&searchFilter=
Image 5-1 -
You need to enable any items you want to know. In the above page, If you want to change any items' screens, you can click the screens link in the actions column for that item. Then in the next page, check the project you want to change, and update it. Like: Story points
Image 5-2
Image 5-3
For the next-gen Jira, when you add story points item, the name should be Story Points or Story point estimate.
For now, we just can download the code in our local machine, please follow below steps:
- Clone the backend code in your local machine: https://github.com/thoughtworks/HeartBeat/
- Follow the steps as below
$ docker-compose up
Install node modules:
cd HearBeat/backend
yarn installRun backend:
yarn startswagger address: http://localhost:3001/swagger-html
yarn packageyou can build server to binary file, it will output 3 files:
- heartbeat-backend-linux
- heartbeat-backend-macos
- heartbeat-backend-win.exe
these files can run on different system, you do not need to install node environment
cd HearBeat/frontend
yarn install --pure-lockfile
yarn start
yarn build