Example REST API application definition for GitLab data import.
Steps to add the application to your eazyBI:
- Import the following application:
REST API application definition
{
"application_type": "rest_api",
"application_params": {
"source_params": {
"url": "http://gitlab.local:8001/api/v4/projects/3/merge_requests?order_by=updated_at",
"pagination": "next_page_url",
"incremental": true,
"incremental_stop_import": true,
"authentication_type": "oauth2",
"content_type": "json",
"custom_javascript_code": "var results = []\nvar mrIssueKeys = []\n//Change to the actual issue key expression\nregex = /CA-\\d*/gi\n//Change to the actual Jira project key\njira_project_key = 'CA'\n//Change to the actual Gitlab project ID\nproject_id = 3\n\n//Add issue reference to a merge request. Check that each issue is added only once\nfunction addMRResult(issue_key) {\n if(!_.includes(mrIssueKeys, issue_key)) {\n results.push({\n source_id: doc.id.toString() + \"-\" + issue_key,\n project_key: jira_project_key,\n issue_key: issue_key,\n merge_request_merged: doc.merged_at,\n merge_request_created: doc.created_at\n })\n mrIssueKeys.push(issue_key)\n }\n}\n\n//Check issue references in MR base information\nissue_keys = doc.title.match(regex) || [] \nissue_keys = issue_keys.concat(doc.source_branch.match(regex))\n_.map(_.compact(_.uniq(issue_keys)), function(issue_key) {\n addMRResult(issue_key)\n})\n\n//Get issue references from commit messages\ngetDocument(\"/api/v4/projects/\" + project_id +\"/merge_requests/\" + doc.iid + \"/commits\", function(commits) {\n _.each(commits, function(commit) {\n issue_keys = commit.title.match(regex) || []\n issue_keys = issue_keys.concat(commit.message.match(regex))\n _.map(_.compact(_.uniq(issue_keys)), function(issue_key) {\n addMRResult(issue_key)\n //Add commit created time\n results.push({\n source_id: doc.id.toString() + \"-COMMIT-\" + commit.id.toString() + \"-\" + issue_key,\n project_key: jira_project_key,\n issue_key: issue_key,\n commit_created: commit.created_at\n })\n })\n })\n})\n//Get issue references from comments\ngetDocument(\"/api/v4/projects/\" + project_id +\"/merge_requests/\" + doc.iid + \"/notes\", function(comments) {\n _.each(comments, function(comment) {\n //Ignore system automatically generated comments\n if (comment.system) {\n return\n }\n issue_keys = comment.body.match(regex)\n _.map(_.compact(_.uniq(issue_keys)), function(issue_key) {\n addMRResult(issue_key)\n //Add comment created time\n results.push({\n source_id: doc.id.toString() + \"-COMMENT-\" + comment.id.toString() + \"-\" + issue_key,\n project_key: jira_project_key,\n issue_key: issue_key,\n comment_created: comment.created_at\n })\n })\n })\n})\n\nreturn results;",
"oauth2_client_id": "e4bd16e838398505f1f3a4d6d8a39947575afb3a62122fffe0734e891f7d6f39",
"oauth2_client_secret": "c90dff2c932207cec874e593be0ab36e4bc23c64b0a280d501c18a9e279d5150",
"oauth2_authorize_url": "http://gitlab.local:8001/oauth/authorize",
"oauth2_token_url": "http://gitlab.local:8001/oauth/token",
"oauth2_scope": "api",
"incremental_additional_url_parameters": "updated_after={{ 3 days ago | %Y-%m-%d }}"
},
"extra_options": {
"regular_import_frequency": 0,
"regular_import_at": "",
"time_zone": "Helsinki"
}
},
"source_cube_name": "Issues",
"columns": [
{
"name": "source_id",
"data_type": "string",
"source_id_column": true
},
{
"name": "project_key",
"data_type": "string",
"dimension": "Project",
"key_column": true
},
{
"name": "issue_key",
"data_type": "string",
"dimension": "Issue",
"key_column": true,
"skip_missing": true
},
{
"name": "merge_request_merged",
"data_type": "datetime",
"dimension": "Issue",
"property": "MR merged",
"date_count_measure": "MRs merged",
"date_count_dimension": "Time"
},
{
"name": "merge_request_created",
"data_type": "datetime",
"dimension": "Issue",
"property": "MR created",
"date_count_measure": "MRs created",
"date_count_dimension": "Time"
},
{
"name": "commit_created",
"data_type": "datetime",
"dimension": "Issue",
"property": "Commit created",
"date_count_measure": "Commits created",
"date_count_dimension": "Time"
},
{
"name": "comment_created",
"data_type": "datetime",
"dimension": "Issue",
"property": "Comment created",
"date_count_measure": "Comments created",
"date_count_dimension": "Time"
}
]
}
- Change text <GITLAB_URL> with your actual GitLab URL.
- Define eazyBI OAuth2 application inside GitLab:
3.1) Open the following URL <GITLAB_URL>/admin/applications
3.2) Provide “Redirect URL”. It can be found in the eazyBI REST API application screen and will be something similar to this - <JIRA_BASE_URL>/plugins/servlet/eazybi/source_applications/auth/rest_api/callback
3.3) Grant access to REST API - check the checkbox for “api” scope. It says that you will grant complete read/write access to the API, including all groups and projects. eazyBI for sure will use only read-only access.
3.4) After saving new application you will be provided by Application ID and Secret. Copy first one in Client ID field in eazyBI REST API definition and second in Client secret field. - In eazyBI REST API click on “Edit custom JavaScript code” link and change the following lines corresponding to your settings:
4.1) Change to the actual issue key expression:
regex = /CA-\d*/gi
4.2) Change to the actual Jira project key
jira_project_key = 'CA'
4.3) Change to the actual Gitlab project ID. It can be found by opening the following URL - <GITLAB_URL>/api/v4/projects
project_id = 3
- In eazyBI click “Continue” and you should be provided by sample data set and then click “Start import”.
Each next import will be performed incrementally.
There will be created the following measures:
- MRs created
- MRs merged
- Commits created
- Comments created
Those measures will be tied to Project and Issues dimensions.