Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Changes for score collector for dashboard #1

Open
wants to merge 37 commits into
base: master
Choose a base branch
from

Conversation

sidbha007
Copy link

@sidbha007 sidbha007 commented Feb 2, 2018

Feature to score team dashboards

  • Overall score for a dashboard between 0-5
  • Score for individual widget data: build, scm, quality, deploy

Changes include

  • Score collector to calculate score for enabled dashboards (collectors/misc/score-collector)
  • UI widget to display and configure score
  • Core models, repositories & API changes

Score Collector:

  • Collector will run periodically to calculate score for enabled dashboards
  • Configurations for score calculation can be set at default and individual widgets
  • Configurable criteria to set score if it meets a threshold.

Flow to view score for a dashboard

  1. Enable score widget by clicking on configure and save

score_view_config

  1. After score collector has run, you should see overall score for the team dashboard

score_view

  1. Details for score can be viewed by clicking View Details

score_view_details

  1. Individual widgets will have score view, and clicking on it will show details of the score

score_widget_view

@CLAassistant
Copy link

CLAassistant commented Feb 2, 2018

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ sidbha007
❌ Tapabrata "Topo" Pal


Tapabrata "Topo" Pal seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

sidbha007 and others added 28 commits February 16, 2018 13:17
Core
-  Add ScoreValueType enum, in future if score is calculated for different types like Team, User
- Update models to have map options for additional data

Score Collector
- Fix score tests
- Fix score calculation for builds
- Remove options specific to walmart for scoring in scm and deploy
UI
- Add changes in capitalone template
- Fixes to view scoring
- remove unused variables and code
- remove settings type and custom config settings as it is no longer use
- Made model names generic with Score`Component`Metric as name.
- Add refId property in ScoreComponentMetrics to have reference Ids. This would have widgetId reference when score is for dashboard
- Change `id, name` properties to `displayId, displayName`
- Use refId to show score for a widget in UI
- other cleanup
- mongodb collection score_criteria_settings to save settings
- refactor and move settings classes to core
- changes to save criteria settings on collector run
- Add properties for scoreEnabled and scoreDisplay in dashboard request.
- ScoreDashboardService to associate collectorItem for score dashboard
- Fix tests
- UI changes in create/edit dashboard screen to enable/disable scoring
- API to update enable/disable scoring of dashboard
- directive to display widget in header
- api to fetch score by dashboard id
- common score repository
- directive to view in widget
- update in templates
- show score as first item in custom/widget templates
- fixes
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
)

* Merge from upstream (#1)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Change in log file name

* Adding README.md for rally collector

* Rally URL change

* usernames and passwords typo change

* Rally username and password details

* Redirect README.md to gh-pages

* fix generics eror: type 1

cast each Evalutor to remove generic information to enable build on some jvm

* Hygieia publisher changes - publish to multiple hygieia instances

* Implementing timeOffset for Incidents and ChangeOrders, and using service name for indients when affectedItem is not available

* Added logic to consider the environment field value on hpsm, and also added the unit tests

* HpsmSettings, changed variable name, new name: incidentEnvironments. And brought down the log-level to debug for hpsmIncidentUpdates collector

* Fixing a minor error on the unit test that was not apparent, due to date when I tested it last

* Version increment for hygieia jenkins plugin (hygieia#2445)

* Updating model to revert change and add comment

* Adding commonName field compare

*  Publish events from Jenkins Pipeline to Hygieia and increment version. (hygieia#2452)

* Version increment for hygieia jenkins plugin

* Publish events from Jenkins Pipeline to Hygieia.

* Publish events from Jenkins Pipeline to Hygieia and increment version.

* Remove SpringBeans

* Removing some whitespaces and proper indendation

* Increasing Version to 2.1.3-SNAPSHOT

* Enhanced Jira feature collector with issueLinks (hygieia#2448)

* issue links details for Feature Collector

* collecting IssueLinks data for a Jira Issue

collecting IssueLinks data for a Jira Issue

* adding IssueLinkType to IssueLinks in Feature collection

* adding comments

* cleanup for issueLinkType data in feature collection

* removing unused import

* removing the _id part from the issueLinks

* fixing banner

* quick fix for the unit tests for fetaure collector

* unit tests for issueLinks

* updates to test data

* New features to sync dashboard based on collector item relations. New API to create Generic collector item

* New features to sync dashboard based on collector item relations. New API to create Generic collector item - adding missing files

* New features to sync dashboard based on collector item relations. New API to create Generic collector item - adding more missing files

* New features to sync dashboard based on collector item relations. Correct test classes.

* New feature to allow creation of tool specific collector item.
Changes to jenkins plugin. Removed MultiSCM support.

* New feature to allow creation of tool specific collector item.
Changes to jenkins plugin. Removed MultiSCM support.

* Reverting some changes to fix test cases.

* Refacrtoring, test updates.

* Added processing time flag for generic collector item. More clean up

* New repository method to look up by processing time.

* New repository method to look up by processing time.
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
* Swagger changes for Audit API; redirect score collector README (hygieia#2129)

* Access private repos with token, removed logging (hygieia#2131)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Variables and their default are separated with : (hygieia#2122)

* Git collector continues to collect data and skip error  (hygieia#2138)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Dashboard Type Filtering over Title Filtering with Pagination (hygieia#2004)

* Changes to incorporate dashboard-type filtering along with title-based filtering

* Some minor fixes based on local testing

* Removed @requestbody from input  (hygieia#2150)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Removed @requestbody for input

* Modified Hudson collector - push based call (hygieia#2152)

* Fix Docker build process, score-properties-builder.sh was not valid, hygeia-jenkins-plugin failed due to mismatched dependency (hygieia#2113)

* Audit check for commits with Service account  (hygieia#2157)

* Added feature to capture LDAP_DN from Github. Added feature to detect Service Account activity in audit

* Add MergeEvent.java.

* Add COMMIT_WITH_SYSACC to direct commit

* removed junk (#2156)

* Accept personalAccessToken at repo level (hygieia#2159)

* Fix for Target and Achieved (hygieia#2165)

Testcase steps array have data in the following order Target, achieved. Currently the code is always updating Achieved property with "0" and target property with the achieved value. This fix should set target property with target value and achieved property with achieved value.

* Missing properties in properties-builder scripts See hygieia#2154 (hygieia#2168)

* UI changes for General Configuration (hygieia#2073)

* UI changes for General Congifuration

* Fixes for the pull request hygieia#2073

* Audit api - Test cases for system id check  (hygieia#2175)

* Accept personalAccessToken at repo level

* Testcase for systemId check fix

* Configuration option in admin page with collector changes & Rally collector initial commit (hygieia#2074)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Update Node version to 6.9.1 (hygieia#2179)

* UI changes for rally widget (hygieia#2085)

* UI changes for rally widget

* Removed logger from the file

* UI changes for making Rally widget as standalone

* Remove company specific information in Rally widget view.js

* Change in log file name

* Adding README.md for rally collector

* Rally URL change

* usernames and passwords typo change

* Rally username and password details

* Redirect README.md to gh-pages

* Collect Static Security Analysis from eratocode (hygieia#2184)

* Collect Static Security Analysis from eratocode

* fixed static date range to avoid period if 60 days

* Added test scenario for High and Critical code scan analysis.

* Removed unused data.

* Update README.md (hygieia#2209)

* Upgrade NPM version to fix UI build issues for new users (hygieia#2217)

* Upgrade NPM version to fix UI build issues for new users

* Delete package-lock.json

* Delete package-lock.json

* fixes issue with configItem being null for create dashboard (hygieia#2228)

* Adding Environments field to cmdb model (hygieia#2234)

* Correct parsing of the MONGO_PORT variable (hygieia#2123)

* Fix for hygieia#2246, changing npm url from http to https (hygieia#2248)

* Adding library policy (open source scan) audit api. (hygieia#2240)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account  (hygieia#2251)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Fixing issue hygieia#1084. Now the collector works with UrbanCode 6.2.*. It might not be backward compatible with previous versions. (hygieia#1515)

* adding updated sonar interface to fetch for Sonar 5.6.7 with more than 500 projects (hygieia#2243)

* adding updated sonar interface

* removing error logging

* Infinte loop (hygieia#2124)

* Fix TypeError: Cannot read property 'name' of undefined

hygieia#1599 (comment)

* Bug: Login infinite loop

If token is revoked server side, app goes into infinite loop due to login controller thinking user is authed and redirecting.

if (userService.isAuthenticated()) {
            $location.path('/');
            return;
        }

* Travis build notifications modified (hygieia#2256)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Notifications modified

* Notifications modified

* Fixed the errors with npm install of phantom-prebuilt (hygieia#2253)

* Revert "Fixed the errors with npm install of phantom-prebuilt (hygieia#2253)" (hygieia#2258)

This reverts commit 540764f.

*  Fixed the errors with npm install of phantom-prebuilt for first time users. (hygieia#2263)

* Fixed the errors with npm install of phantom-prebuilt

* Delete package-lock.json

* update pom.xml added --ignore-scripts to npm

Added install --ignore-scripts to npm task to fix issues with phantomjs-prebuilt

* Update .gitignore to ignore package-lock.json

Added package-lock.json to .gitignore so that it does not get committed and no violations henceforth

* fix generics eror: type 1

cast each Evalutor to remove generic information to enable build on some jvm

* Sonar 6.x - fetch projects on page size (hygieia#2268)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Notifications modified

* Notifications modified

* Sonar6.x - fetch projects on page size

* Sonar6.x - fetch projects on page size

* Code_Quality review modified- quality_gate_metrics (hygieia#2279)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Notifications modified

* Notifications modified

* Sonar6.x - fetch projects on page size

* Sonar6.x - fetch projects on page size

* Code_Quality review modified- quality_gate_metrics logic

* Code_Quality review modified- quality_gate_metrics logic

* add in optional filters into describe instances for aws cloud collector + region support (hygieia#2264)

* add in optional filters into describe instances for aws cloud collector

it may be that you do not want to look at all instances in an account,
but a smaller more focussed list. To do this you can now use filtering on
the aws side to describe only the instances that match your filter.

* ensure filter maps is constructed

* add in region support

* remove committed jar (hygieia#2283)

* Peer review audit - Direct commits to base (hygieia#2284)

* Fix for KPI description for perf audit API. (hygieia#2288)

* Revert "Fix for hygieia#2246, changing npm url from http to https" (hygieia#2252)

* Revert "Adding library policy (open source scan) audit api. (hygieia#2240)"

This reverts commit 8257ef7.

* Revert "Fix for hygieia#2246, changing npm url from http to https (hygieia#2248)"

This reverts commit 68e273a.

* Update for plugin to store build path instead of name (hygieia#2306)

* Update for plugin to store build path instead of name

* Updating to return job name if something is wrong with url

* Removing whitespace

* Fix dashboard rename bug (hygieia#2315)

* Fixes with slowness when renaming dashboard

* Fix for broken build

* Centralized entreprise certificate management + addition in Bitbucket collector (hygieia#2162)

* Copy certs in image

* Mount certs volume and import its content in bitbucket image

* Readme and script that allows centralized crt storage and addition in the bitbucket collector

* better looking readme

* include env var in the procedure

* skip build docker for rally (hygieia#2314)

* adding new audit staus for SCA Audit API

* Changes to generate collector items for the incidents (hygieia#2316)

* Changes to generate collector items for incidents

* A minor clean up

* A minor cleanup

* A minor clean up

* Null and Empty check on options, that causing an NPE

* Removed class level variable declaration for pageOfCollectorItems that was causing a pmd checks failure for not being used

* Changed the setters and getters for open and closed time to Long type as I was getting an NPE when calling the getters when there was no value in the database

* A minor update to handle closedTime

* Changed the setter method for open and closed time to call other ovewrloaded setter method

* HotHouse changes implementation to codeQualityEvaluator

* adding audit status for CODE_QUALITY_CHECK_IS_CURRENT

* fongo test data for SCA

* updating chrome driver to latest

* fongo test data for SCA

* Fongo unit tests from Praveen

* Gates Found Audit Status

* updating it to sorted set

* Fix for performance api to pass in metrics (hygieia#2339)

* Fix for performance api to pass in metrics

* Adding Target App and Env

* Api audit - SCA metrics fix for 6.7 (hygieia#2348)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* updates to codeQualityEvaluator

* CollectorItem refresh (hygieia#2356)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* CollectorItem refresh
Sonar Collector - remove projectId as unique field

* Adding logic to update incidents with the latest data from hpsm (hygieia#2345)

* Adding logic to update incidents with the latest data from hpsm

* Removed paging logic, fetching updates for incidents with collectorItems attached to dashboards

* Removed paging logic, fetching updates for incidents with collectorItemIds attached to dashboards

* Removed IncidentUpdatesRepository.java that was previously used with the paging logic

* Added methods to support paging

* Fixed a bug in the github client that improperly handled paged responses (hygieia#2312)

Github's `Link` header values are URL Encoded. An example:

```
Key: Link
Value: <https://api.github.com/repositories/<some_repo>/commits?sha=refs%2Fheads%2Fmaster&since=2018-06-06T17%3A39Z&page=2>; rel="next", <https://api.github.com/repositories/<some_repo>/commits?sha=refs%2Fheads%2Fmaster&since=2018-06-06T17%3A39Z&page=2>; rel="last"
```

The client as written is incapable of correctly following an encoded link and results in a 404 Not Found error.

I suspect this error is what has been reported here and there already but hasn't been reproduced -- I spent the day spelunking this project's Issue history and noticed several issues similar to what I observed today.

I confirmed this resolution by crafting a small test I could step through in my IDE, that looked like:

```
    @test
    public void breakingScenario() throws MalformedURLException, HygieiaException {

        GitHubRepo repo = new GitHubRepo();
        repo.setRepoUrl("https://github.com/<my organization>/<a high velocity repository>");
        repo.setBranch("refs/heads/master");
        repo.setPersonalAccessToken("<redacted>");

        GitHubSettings settings = new GitHubSettings();
        settings.setErrorThreshold(100);
        settings.setKey("<redacted>");
        GitHubClient client = new DefaultGitHubClient(settings, new RestOperationsSupplier());

        List<Commit> commits = client.getCommits(repo, true, new ArrayList<>());
        assert commits.size() > 0;
    }
```

On master, this test passes for repositories returning no `Link` header, but fails on repositories returning a `Link` header.
I was able to force the test to fail by amending my `GithubSettings` to search a larger trailing window, say 100 days instead of 14,
as this ensured for any reasonably active repository that I'd see paged results.

After applying my decoding change, the test passed for all repositories I tried, and for all date ranges.

* adding audit statuses for SCA thresholds found

* added the thresholds found audit statuses

* update to test data

* unit tests for the new audit statuses

* adding status for thresholds found. reading the metrics from an enum

* Changed the datatype of 'CodeQualityMetric' 'value' variable to String, from Object (hygieia#2369)

* Remove unused dependencies from AWS Cloud Collector Microservice (hygieia#2364)

* Remove unused dependencies from pom.xml

* Typo

* audit-api SCA audit status & remote rename dashboard (hygieia#2382)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* CollectorItem refresh
Sonar Collector - remove projectId as unique field

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* updating expected result for SCA

* test commit pus

* Revert "test commit pus"

This reverts commit 406ba05.

* cleaning up the data

* cleanup for test data

* Model Changes and UniqueCriteria (hygieia#2390)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* test data cleanup

* Nexus IQ scan enhancements (hygieia#2391)

* Merging latest changes from master (#5)

* Fix sonar CodeQuallity collector parse error (hygieia#2028) (hygieia#2092)

* Fix TypeError: Cannot read property 'name' of undefined (hygieia#2094)

hygieia#1599 (comment)

* Chown in UI dockerfiles to change owner on nginx files (hygieia#2108)

* Chown in UI dockerfiles to change owner on nginx files

* removed useless comand

* Set DB properties to the correct defaults in all collectors (hygieia#2107)

* Mismatch between Dockerfiles and properties builder script on file names

* default DB names changed to dashboarddb

* default DB usernames changed to dashboarduser

* default DB passwords changed to dbpassword

* revert modif on score dockerfile

* Exclude merge from branch being evaluated to the branch the PR came from in case of squash merge. (hygieia#2112)

* RegressionTestResultEvaluator Bug Fix (hygieia#2116)

* Access git private repos with personal access token (hygieia#2118)

* Access git private repos with personalAccessToken

* Access git private repos with personalAccessToken

* Access git private repos with personalAccessToken

* Path correction (hygieia#2127)

* Hide label for code repo  (hygieia#2130)

* Path correction

* Hide label

* Swagger changes for Audit API; redirect score collector README (hygieia#2129)

* Access private repos with token, removed logging (hygieia#2131)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Variables and their default are separated with : (hygieia#2122)

* Git collector continues to collect data and skip error  (hygieia#2138)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Dashboard Type Filtering over Title Filtering with Pagination (hygieia#2004)

* Changes to incorporate dashboard-type filtering along with title-based filtering

* Some minor fixes based on local testing

* Removed @requestbody from input  (hygieia#2150)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Removed @requestbody for input

* Modified Hudson collector - push based call (hygieia#2152)

* Fix Docker build process, score-properties-builder.sh was not valid, hygeia-jenkins-plugin failed due to mismatched dependency (hygieia#2113)

* Audit check for commits with Service account  (hygieia#2157)

* Added feature to capture LDAP_DN from Github. Added feature to detect Service Account activity in audit

* Add MergeEvent.java.

* Add COMMIT_WITH_SYSACC to direct commit

* removed junk (#2156)

* Accept personalAccessToken at repo level (hygieia#2159)

* Fix for Target and Achieved (hygieia#2165)

Testcase steps array have data in the following order Target, achieved. Currently the code is always updating Achieved property with "0" and target property with the achieved value. This fix should set target property with target value and achieved property with achieved value.

* Missing properties in properties-builder scripts See hygieia#2154 (hygieia#2168)

* UI changes for General Configuration (hygieia#2073)

* UI changes for General Congifuration

* Fixes for the pull request hygieia#2073

* Audit api - Test cases for system id check  (hygieia#2175)

* Accept personalAccessToken at repo level

* Testcase for systemId check fix

* Configuration option in admin page with collector changes & Rally collector initial commit (hygieia#2074)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Update Node version to 6.9.1 (hygieia#2179)

* UI changes for rally widget (hygieia#2085)

* UI changes for rally widget

* Removed logger from the file

* UI changes for making Rally widget as standalone

* Remove company specific information in Rally widget view.js

* Sonatype UI Enhancement Code changes

* Sontaype UI enhancement code changes

* Nexus IQ scan enhancements

* Removed commented lines of code

* code cleanup

* adding collector items for perf

* addressing review comments from Ragha & Nireesh

* Handle unknown scmAuthor  (hygieia#2394)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* Team widget  (hygieia#2398)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* CollectorItem refresh
Sonar Collector - remove projectId as unique field

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* Team widget

* Team widget

* Team widget

* Parameterized read and connect timeouts (hygieia#2399)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* using StringUtils instead of String equals

* using StringUtils

* cleaning up test data

* Reverse Peer Review Logic - GHR takes precedence over LGTM (hygieia#2406)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* Reverse Peer Review Logic - GHR takes precedence over LGTM

* Handle missing login info in github commits (hygieia#2407)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* Reverse Peer Review Logic - GHR takes precedence over LGTM

* Fix for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when login information is missing from github commit.

* Fixed the docker compose build path of the gitlab scm collector (hygieia#2365)

* updating the condition

* reverting my changes to items.json

* updating items.json from fongo unit tests

* Fix the audit logic (hygieia#2409)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* Reverse Peer Review Logic - GHR takes precedence over LGTM

* Fix for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when login information is missing from github commit.

* Auditing logic refactor

* Audit status handling Git commits without login info (hygieia#2410)

* Handle scenario when PR is misinterpreted leading to wrong Audit Status (hygieia#2411)

* Audit status handling Git commits without login info

* Handle scenario when PR is misinterpreted leading to returning false audit status - DIRECT_COMMITS_BASE as the current collector handles it as New commit.

* Handle scenario when PR is misinterpreted leading to returning false audit status - DIRECT_COMMITS_BASE as the current collector handles it as New commit.

* reverting * import to reflect actual classes

* Traceability of Jira Stories from Test Results

* addressing comments from Ragha

* Hygieia publisher changes - publish to multiple hygieia instances

* Implementing timeOffset for Incidents and ChangeOrders, and using service name for indients when affectedItem is not available

* Added logic to consider the environment field value on hpsm, and also added the unit tests

* HpsmSettings, changed variable name, new name: incidentEnvironments. And brought down the log-level to debug for hpsmIncidentUpdates collector

* Corrected logic to evaluate commits that are on the PR but outside of queried date range. (hygieia#2422)

* Github replaces underscores with hyphens in user account. Changed collector code to handle this. (hygieia#2423)

* Corrected logic to evaluate commits that are on the PR but outside of queried date range.

* Github replaces underscores with hyphen in user accounts. Changed code to handle this.

* Fixing a minor error on the unit test that was not apparent, due to date when I tested it last

*  Update README (hygieia#2095)

Update README to describe how to start a mocked version of the UI to run the tests against

* Performance Hot House changes (hygieia#2401)

* Hothouse changes for performance evaulator

* Fongo unit test for performance evaulator

* cleanup for test data

* update branch with master

* code cleanup

* code cleanup

* Code clean up

* bug fix in performance evaluator

* Fixed the Audit Status for 'error' to PEER_REVIEW_LGTM_ERROR from PEER_REVIEW_LGTM_PENDING

* PT audit changes - Generic collector for Performance tests

* Sonar collector - null check for project data (hygieia#2436)

* recursively chown home

* missed user:group

* Update Readme

Updated to announce the Hygieia Org Move.

* Revert "Merge pull request hygieia#2431 from battinasubodh/master": To avoid downstream impacts at this point. (hygieia#2438)

This reverts commit 7e7cc87, reversing
changes made to c4f329d.

* fix locale to the US for formatted numbers (hygieia#2430)

* Added ability to globally publish build data using Jenkins Hygieia plugin. (hygieia#2396)

* Added ability to globally publish all build data.

* Rename variable to indicate that it is a global setup.

* Added global publish of Sonar result. Also added bunch of test cases.

* upgraded the hygieia publisher plugin version to 2.1.0 (hygieia#2440)

* Updated readme (hygieia#2441)

* updated readme

* dumy build to push repo

* Adding new item to model

* Remove instance variables from the listener. Changed test cases accordingly.

* Version increment for hygieia jenkins plugin (hygieia#2445)

* Updating model to revert change and add comment

* Adding commonName field compare

*  Publish events from Jenkins Pipeline to Hygieia and increment version. (hygieia#2452)

* Version increment for hygieia jenkins plugin

* Publish events from Jenkins Pipeline to Hygieia.

* Publish events from Jenkins Pipeline to Hygieia and increment version.

* Remove SpringBeans

* Removing some whitespaces and proper indendation

* Increasing Version to 2.1.3-SNAPSHOT

* Enhanced Jira feature collector with issueLinks (hygieia#2448)

* issue links details for Feature Collector

* collecting IssueLinks data for a Jira Issue

collecting IssueLinks data for a Jira Issue

* adding IssueLinkType to IssueLinks in Feature collection

* adding comments

* cleanup for issueLinkType data in feature collection

* removing unused import

* removing the _id part from the issueLinks

* fixing banner

* quick fix for the unit tests for fetaure collector

* unit tests for issueLinks

* updates to test data

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Give the github-graphql docker image a unique name, Fixes hygieia#2459 (hygieia#2460)

* Widget display on lastUpdated  (hygieia#2458)

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - get last updated test run  (hygieia#2467)

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - NPE check for test list

* Sca verbose bug 21485 (hygieia#2450)

* bug fix for `SCA Verbose Message not returning SCA Project URL consistently`

* cleanup

* covering for new coverage

* updating comments

* reverting items.json

* updating items.json

* addressing Nireesh comment for replacing nested catch with multi catch block

* SCA Unit Tests

* code cleanup

* New Feature: Auto complete dashboard by correlating events (hygieia#2466)

* Merge from upstream (#1)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Change in log file name

* Adding README.md for rally collector

* Rally URL change

* usernames and passwords typo change

* Rally username and password details

* Redirect README.md to gh-pages

* fix generics eror: type 1

cast each Evalutor to remove generic information to enable build on some jvm

* Hygieia publisher changes - publish to multiple hygieia instances

* Implementing timeOffset for Incidents and ChangeOrders, and using service name for indients when affectedItem is not available

* Added logic to consider the environment field value on hpsm, and also added the unit tests

* HpsmSettings, changed variable name, new name: incidentEnvironments. And brought down the log-level to debug for hpsmIncidentUpdates collector

* Fixing a minor error on the unit test that was not apparent, due to date when I tested it last

* Version increment for hygieia jenkins plugin (hygieia#2445)

* Updating model to revert change and add comment

* Adding commonName field compare

*  Publish events from Jenkins Pipeline to Hygieia and increment version. (hygieia#2452)

* Version increment for hygieia jenkins plugin

* Publish events from Jenkins Pipeline to Hygieia.

* Publish events from Jenkins Pipeline to Hygieia and increment version.

* Remove SpringBeans

* Removing some whitespaces and proper indendation

* Increasing Version to 2.1.3-SNAPSHOT

* Enhanced Jira feature collector with issueLinks (hygieia#2448)

* issue links details for Feature Collector

* collecting IssueLinks data for a Jira Issue

collecting IssueLinks data for a Jira Issue

* adding IssueLinkType to IssueLinks in Feature collection

* adding comments

* cleanup for issueLinkType data in feature collection

* removing unused import

* removing the _id part from the issueLinks

* fixing banner

* quick fix for the unit tests for fetaure collector

* unit tests for issueLinks

* updates to test data

* New features to sync dashboard based on collector item relations. New API to create Generic collector item

* New features to sync dashboard based on collector item relations. New API to create Generic collector item - adding missing files

* New features to sync dashboard based on collector item relations. New API to create Generic collector item - adding more missing files

* New features to sync dashboard based on collector item relations. Correct test classes.

* New feature to allow creation of tool specific collector item.
Changes to jenkins plugin. Removed MultiSCM support.

* New feature to allow creation of tool specific collector item.
Changes to jenkins plugin. Removed MultiSCM support.

* Reverting some changes to fix test cases.

* Refacrtoring, test updates.

* Added processing time flag for generic collector item. More clean up

* New repository method to look up by processing time.

* New repository method to look up by processing time.

* Updated to give ability to insert a Component into CMDB (hygieia#2472)

* Updates for api to insert new cmdb item

* Adding some comments

* Add Ability to skip jobs from publishing data through Global Configuration (hygieia#2477)

* Added ability to exclude job names

* Added ability to exclude job names

* merge from upstream

* fix mongoDB script interpretation issue (hygieia#2367)

* All tokens to be used for SonarQube authentication

Per the docs for SonarQube at https://docs.sonarqube.org/display/DEV/Web+API when using token based authentication then the token goes into the username part, followed by a colon and empty password.

The current code base does not allow a null password, it is expecting both a username and password, this PR corrects that.

See https://docs.sonarqube.org/display/DEV/Web+API

* Corrected operator precedence

* Set collectorItemId - DeployServiceImpl

*  Audit API - set `SCA CODE_QUALITY_AUDIT_OK` for Metric Status Ok/Warn (hygieia#2484)

* set SCA CODE_QUALITY_AUDIT_OK for Metric Status Ok/Warn

* SCA set CODE_QUALITY_AUDIT_OK when status is OK/Warn

* Nicename - Sonar collector (hygieia#2488)

* Nicename - Sonar collector

* Update pom.xml (hygieia#2427)

frontend-maven-plugin  issue

* NFRR Audit Collector and API Implementation v1

* code formatting

* NFRR API Updates, adding Pagination support.

* NFRR module version updated

* NFRR pom.xml updated - hygieia version to 3.0.1

* PR Review Comments Updates

* PR Review Comments Updates

* NFRR Code Review Updates
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
* Fix sonar CodeQuallity collector parse error (hygieia#2028) (hygieia#2092)

* Fix TypeError: Cannot read property 'name' of undefined (hygieia#2094)

hygieia#1599 (comment)

* Chown in UI dockerfiles to change owner on nginx files (hygieia#2108)

* Chown in UI dockerfiles to change owner on nginx files

* removed useless comand

* Set DB properties to the correct defaults in all collectors (hygieia#2107)

* Mismatch between Dockerfiles and properties builder script on file names

* default DB names changed to dashboarddb

* default DB usernames changed to dashboarduser

* default DB passwords changed to dbpassword

* revert modif on score dockerfile

* Exclude merge from branch being evaluated to the branch the PR came from in case of squash merge. (hygieia#2112)

* RegressionTestResultEvaluator Bug Fix (hygieia#2116)

* Access git private repos with personal access token (hygieia#2118)

* Access git private repos with personalAccessToken

* Access git private repos with personalAccessToken

* Access git private repos with personalAccessToken

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Path correction (hygieia#2127)

* Hide label for code repo  (hygieia#2130)

* Path correction

* Hide label

* Swagger changes for Audit API; redirect score collector README (hygieia#2129)

* Access private repos with token, removed logging (hygieia#2131)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Variables and their default are separated with : (hygieia#2122)

* Git collector continues to collect data and skip error  (hygieia#2138)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Dashboard Type Filtering over Title Filtering with Pagination (hygieia#2004)

* Changes to incorporate dashboard-type filtering along with title-based filtering

* Some minor fixes based on local testing

* Removed @requestbody from input  (hygieia#2150)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Removed @requestbody for input

* Modified Hudson collector - push based call (hygieia#2152)

* Fix Docker build process, score-properties-builder.sh was not valid, hygeia-jenkins-plugin failed due to mismatched dependency (hygieia#2113)

* Audit check for commits with Service account  (hygieia#2157)

* Added feature to capture LDAP_DN from Github. Added feature to detect Service Account activity in audit

* Add MergeEvent.java.

* Add COMMIT_WITH_SYSACC to direct commit

* removed junk (#2156)

* Accept personalAccessToken at repo level (hygieia#2159)

* Fix for Target and Achieved (hygieia#2165)

Testcase steps array have data in the following order Target, achieved. Currently the code is always updating Achieved property with "0" and target property with the achieved value. This fix should set target property with target value and achieved property with achieved value.

* Missing properties in properties-builder scripts See hygieia#2154 (hygieia#2168)

* UI changes for General Configuration (hygieia#2073)

* UI changes for General Congifuration

* Fixes for the pull request hygieia#2073

* Audit api - Test cases for system id check  (hygieia#2175)

* Accept personalAccessToken at repo level

* Testcase for systemId check fix

* Configuration option in admin page with collector changes & Rally collector initial commit (hygieia#2074)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Update Node version to 6.9.1 (hygieia#2179)

* UI changes for rally widget (hygieia#2085)

* UI changes for rally widget

* Removed logger from the file

* UI changes for making Rally widget as standalone

* Remove company specific information in Rally widget view.js

* Change in log file name

* Adding README.md for rally collector

* Rally URL change

* usernames and passwords typo change

* Rally username and password details

* Redirect README.md to gh-pages

* Collect Static Security Analysis from eratocode (hygieia#2184)

* Collect Static Security Analysis from eratocode

* fixed static date range to avoid period if 60 days

* Added test scenario for High and Critical code scan analysis.

* Removed unused data.

* Update README.md (hygieia#2209)

* Upgrade NPM version to fix UI build issues for new users (hygieia#2217)

* Upgrade NPM version to fix UI build issues for new users

* Delete package-lock.json

* Delete package-lock.json

* fixes issue with configItem being null for create dashboard (hygieia#2228)

* Adding Environments field to cmdb model (hygieia#2234)

* Correct parsing of the MONGO_PORT variable (hygieia#2123)

* Fix for hygieia#2246, changing npm url from http to https (hygieia#2248)

* Adding library policy (open source scan) audit api. (hygieia#2240)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account  (hygieia#2251)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Fixing issue hygieia#1084. Now the collector works with UrbanCode 6.2.*. It might not be backward compatible with previous versions. (hygieia#1515)

* adding updated sonar interface to fetch for Sonar 5.6.7 with more than 500 projects (hygieia#2243)

* adding updated sonar interface

* removing error logging

* Infinte loop (hygieia#2124)

* Fix TypeError: Cannot read property 'name' of undefined

hygieia#1599 (comment)

* Bug: Login infinite loop

If token is revoked server side, app goes into infinite loop due to login controller thinking user is authed and redirecting.

if (userService.isAuthenticated()) {
            $location.path('/');
            return;
        }

* Travis build notifications modified (hygieia#2256)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Notifications modified

* Notifications modified

* Fixed the errors with npm install of phantom-prebuilt (hygieia#2253)

* Revert "Fixed the errors with npm install of phantom-prebuilt (hygieia#2253)" (hygieia#2258)

This reverts commit 540764f.

*  Fixed the errors with npm install of phantom-prebuilt for first time users. (hygieia#2263)

* Fixed the errors with npm install of phantom-prebuilt

* Delete package-lock.json

* update pom.xml added --ignore-scripts to npm

Added install --ignore-scripts to npm task to fix issues with phantomjs-prebuilt

* Update .gitignore to ignore package-lock.json

Added package-lock.json to .gitignore so that it does not get committed and no violations henceforth

* fix generics eror: type 1

cast each Evalutor to remove generic information to enable build on some jvm

* Sonar 6.x - fetch projects on page size (hygieia#2268)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Notifications modified

* Notifications modified

* Sonar6.x - fetch projects on page size

* Sonar6.x - fetch projects on page size

* Code_Quality review modified- quality_gate_metrics (hygieia#2279)

* Audit API -Peer Review - Direct commits with Increment version tag by Service Account

* Peer review - Direct commits audit for increment version tag - refactored.

* Notifications modified

* Notifications modified

* Sonar6.x - fetch projects on page size

* Sonar6.x - fetch projects on page size

* Code_Quality review modified- quality_gate_metrics logic

* Code_Quality review modified- quality_gate_metrics logic

* add in optional filters into describe instances for aws cloud collector + region support (hygieia#2264)

* add in optional filters into describe instances for aws cloud collector

it may be that you do not want to look at all instances in an account,
but a smaller more focussed list. To do this you can now use filtering on
the aws side to describe only the instances that match your filter.

* ensure filter maps is constructed

* add in region support

* remove committed jar (hygieia#2283)

* Peer review audit - Direct commits to base (hygieia#2284)

* Fix for KPI description for perf audit API. (hygieia#2288)

* Revert "Fix for hygieia#2246, changing npm url from http to https" (hygieia#2252)

* Revert "Adding library policy (open source scan) audit api. (hygieia#2240)"

This reverts commit 8257ef7.

* Revert "Fix for hygieia#2246, changing npm url from http to https (hygieia#2248)"

This reverts commit 68e273a.

* Update for plugin to store build path instead of name (hygieia#2306)

* Update for plugin to store build path instead of name

* Updating to return job name if something is wrong with url

* Removing whitespace

* Fix dashboard rename bug (hygieia#2315)

* Fixes with slowness when renaming dashboard

* Fix for broken build

* Centralized entreprise certificate management + addition in Bitbucket collector (hygieia#2162)

* Copy certs in image

* Mount certs volume and import its content in bitbucket image

* Readme and script that allows centralized crt storage and addition in the bitbucket collector

* better looking readme

* include env var in the procedure

* skip build docker for rally (hygieia#2314)

* adding new audit staus for SCA Audit API

* Changes to generate collector items for the incidents (hygieia#2316)

* Changes to generate collector items for incidents

* A minor clean up

* A minor cleanup

* A minor clean up

* Null and Empty check on options, that causing an NPE

* Removed class level variable declaration for pageOfCollectorItems that was causing a pmd checks failure for not being used

* Changed the setters and getters for open and closed time to Long type as I was getting an NPE when calling the getters when there was no value in the database

* A minor update to handle closedTime

* Changed the setter method for open and closed time to call other ovewrloaded setter method

* HotHouse changes implementation to codeQualityEvaluator

* adding audit status for CODE_QUALITY_CHECK_IS_CURRENT

* fongo test data for SCA

* updating chrome driver to latest

* fongo test data for SCA

* Fongo unit tests from Praveen

* Gates Found Audit Status

* updating it to sorted set

* Fix for performance api to pass in metrics (hygieia#2339)

* Fix for performance api to pass in metrics

* Adding Target App and Env

* Api audit - SCA metrics fix for 6.7 (hygieia#2348)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* updates to codeQualityEvaluator

* CollectorItem refresh (hygieia#2356)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* CollectorItem refresh
Sonar Collector - remove projectId as unique field

* Adding logic to update incidents with the latest data from hpsm (hygieia#2345)

* Adding logic to update incidents with the latest data from hpsm

* Removed paging logic, fetching updates for incidents with collectorItems attached to dashboards

* Removed paging logic, fetching updates for incidents with collectorItemIds attached to dashboards

* Removed IncidentUpdatesRepository.java that was previously used with the paging logic

* Added methods to support paging

* Fixed a bug in the github client that improperly handled paged responses (hygieia#2312)

Github's `Link` header values are URL Encoded. An example:

```
Key: Link
Value: <https://api.github.com/repositories/<some_repo>/commits?sha=refs%2Fheads%2Fmaster&since=2018-06-06T17%3A39Z&page=2>; rel="next", <https://api.github.com/repositories/<some_repo>/commits?sha=refs%2Fheads%2Fmaster&since=2018-06-06T17%3A39Z&page=2>; rel="last"
```

The client as written is incapable of correctly following an encoded link and results in a 404 Not Found error.

I suspect this error is what has been reported here and there already but hasn't been reproduced -- I spent the day spelunking this project's Issue history and noticed several issues similar to what I observed today.

I confirmed this resolution by crafting a small test I could step through in my IDE, that looked like:

```
    @test
    public void breakingScenario() throws MalformedURLException, HygieiaException {

        GitHubRepo repo = new GitHubRepo();
        repo.setRepoUrl("https://github.com/<my organization>/<a high velocity repository>");
        repo.setBranch("refs/heads/master");
        repo.setPersonalAccessToken("<redacted>");

        GitHubSettings settings = new GitHubSettings();
        settings.setErrorThreshold(100);
        settings.setKey("<redacted>");
        GitHubClient client = new DefaultGitHubClient(settings, new RestOperationsSupplier());

        List<Commit> commits = client.getCommits(repo, true, new ArrayList<>());
        assert commits.size() > 0;
    }
```

On master, this test passes for repositories returning no `Link` header, but fails on repositories returning a `Link` header.
I was able to force the test to fail by amending my `GithubSettings` to search a larger trailing window, say 100 days instead of 14,
as this ensured for any reasonably active repository that I'd see paged results.

After applying my decoding change, the test passed for all repositories I tried, and for all date ranges.

* adding audit statuses for SCA thresholds found

* added the thresholds found audit statuses

* update to test data

* unit tests for the new audit statuses

* adding status for thresholds found. reading the metrics from an enum

* Changed the datatype of 'CodeQualityMetric' 'value' variable to String, from Object (hygieia#2369)

* Remove unused dependencies from AWS Cloud Collector Microservice (hygieia#2364)

* Remove unused dependencies from pom.xml

* Typo

* audit-api SCA audit status & remote rename dashboard (hygieia#2382)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* CollectorItem refresh
Sonar Collector - remove projectId as unique field

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* updating expected result for SCA

* test commit pus

* Revert "test commit pus"

This reverts commit 406ba05.

* cleaning up the data

* cleanup for test data

* Model Changes and UniqueCriteria (hygieia#2390)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* test data cleanup

* Nexus IQ scan enhancements (hygieia#2391)

* Merging latest changes from master (#5)

* Fix sonar CodeQuallity collector parse error (hygieia#2028) (hygieia#2092)

* Fix TypeError: Cannot read property 'name' of undefined (hygieia#2094)

hygieia#1599 (comment)

* Chown in UI dockerfiles to change owner on nginx files (hygieia#2108)

* Chown in UI dockerfiles to change owner on nginx files

* removed useless comand

* Set DB properties to the correct defaults in all collectors (hygieia#2107)

* Mismatch between Dockerfiles and properties builder script on file names

* default DB names changed to dashboarddb

* default DB usernames changed to dashboarduser

* default DB passwords changed to dbpassword

* revert modif on score dockerfile

* Exclude merge from branch being evaluated to the branch the PR came from in case of squash merge. (hygieia#2112)

* RegressionTestResultEvaluator Bug Fix (hygieia#2116)

* Access git private repos with personal access token (hygieia#2118)

* Access git private repos with personalAccessToken

* Access git private repos with personalAccessToken

* Access git private repos with personalAccessToken

* Path correction (hygieia#2127)

* Hide label for code repo  (hygieia#2130)

* Path correction

* Hide label

* Swagger changes for Audit API; redirect score collector README (hygieia#2129)

* Access private repos with token, removed logging (hygieia#2131)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Variables and their default are separated with : (hygieia#2122)

* Git collector continues to collect data and skip error  (hygieia#2138)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Dashboard Type Filtering over Title Filtering with Pagination (hygieia#2004)

* Changes to incorporate dashboard-type filtering along with title-based filtering

* Some minor fixes based on local testing

* Removed @requestbody from input  (hygieia#2150)

* Path correction

* Hide label

* Access privates repo with personalAccessToken. Removed logging

* Access privates repo with personalAccessToken. Removed logging

* Git collector continues to collect and skip error

* Removed @requestbody for input

* Modified Hudson collector - push based call (hygieia#2152)

* Fix Docker build process, score-properties-builder.sh was not valid, hygeia-jenkins-plugin failed due to mismatched dependency (hygieia#2113)

* Audit check for commits with Service account  (hygieia#2157)

* Added feature to capture LDAP_DN from Github. Added feature to detect Service Account activity in audit

* Add MergeEvent.java.

* Add COMMIT_WITH_SYSACC to direct commit

* removed junk (#2156)

* Accept personalAccessToken at repo level (hygieia#2159)

* Fix for Target and Achieved (hygieia#2165)

Testcase steps array have data in the following order Target, achieved. Currently the code is always updating Achieved property with "0" and target property with the achieved value. This fix should set target property with target value and achieved property with achieved value.

* Missing properties in properties-builder scripts See hygieia#2154 (hygieia#2168)

* UI changes for General Configuration (hygieia#2073)

* UI changes for General Congifuration

* Fixes for the pull request hygieia#2073

* Audit api - Test cases for system id check  (hygieia#2175)

* Accept personalAccessToken at repo level

* Testcase for systemId check fix

* Configuration option in admin page with collector changes & Rally collector initial commit (hygieia#2074)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Update Node version to 6.9.1 (hygieia#2179)

* UI changes for rally widget (hygieia#2085)

* UI changes for rally widget

* Removed logger from the file

* UI changes for making Rally widget as standalone

* Remove company specific information in Rally widget view.js

* Sonatype UI Enhancement Code changes

* Sontaype UI enhancement code changes

* Nexus IQ scan enhancements

* Removed commented lines of code

* code cleanup

* adding collector items for perf

* addressing review comments from Ragha & Nireesh

* Handle unknown scmAuthor  (hygieia#2394)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* Team widget  (hygieia#2398)

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* 1) Fix for quality_gate metrics  6.7
2) Enable api to get sonar project Id for component name.
3) get buss service name and buss app name from dashboard
4) Set project Id as required field in sonar collector

* CollectorItem refresh
Sonar Collector - remove projectId as unique field

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* Audit SCA- add new audit status COLLECTOR_ITEM_ERROR and CODE_QUALITY_NOT_CONFIGURED
GET LDAP user info with personal access token
Rename dashboard for Admin

* Team widget

* Team widget

* Team widget

* Parameterized read and connect timeouts (hygieia#2399)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* using StringUtils instead of String equals

* using StringUtils

* cleaning up test data

* Reverse Peer Review Logic - GHR takes precedence over LGTM (hygieia#2406)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* Reverse Peer Review Logic - GHR takes precedence over LGTM

* Handle missing login info in github commits (hygieia#2407)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* Reverse Peer Review Logic - GHR takes precedence over LGTM

* Fix for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when login information is missing from github commit.

* Fixed the docker compose build path of the gitlab scm collector (hygieia#2365)

* updating the condition

* reverting my changes to items.json

* updating items.json from fongo unit tests

* Fix the audit logic (hygieia#2409)

* Model Changes to have additional metadata

* added unique field criteria as combination of jobName and jobUrl

* added code for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when scmAuthor is unknown

* Update CommonCodeReview.java

removed commented lines

* add Unix to allowed values in test

* Update CodeReviewEvaluatorLegacy.java

* added the review comments for the unknown login

* Update CodeReviewEvaluatorLegacy.java

* parameterized the read and connection timeouts

* Reverse Peer Review Logic - GHR takes precedence over LGTM

* Fix for sending DIRECT_COMMIT_NONCODE_CHANGE_SCM_AUTHOR_LOGIN_INVALID when login information is missing from github commit.

* Auditing logic refactor

* Audit status handling Git commits without login info (hygieia#2410)

* Handle scenario when PR is misinterpreted leading to wrong Audit Status (hygieia#2411)

* Audit status handling Git commits without login info

* Handle scenario when PR is misinterpreted leading to returning false audit status - DIRECT_COMMITS_BASE as the current collector handles it as New commit.

* Handle scenario when PR is misinterpreted leading to returning false audit status - DIRECT_COMMITS_BASE as the current collector handles it as New commit.

* reverting * import to reflect actual classes

* Traceability of Jira Stories from Test Results

* addressing comments from Ragha

* Hygieia publisher changes - publish to multiple hygieia instances

* Implementing timeOffset for Incidents and ChangeOrders, and using service name for indients when affectedItem is not available

* Added logic to consider the environment field value on hpsm, and also added the unit tests

* HpsmSettings, changed variable name, new name: incidentEnvironments. And brought down the log-level to debug for hpsmIncidentUpdates collector

* Corrected logic to evaluate commits that are on the PR but outside of queried date range. (hygieia#2422)

* Github replaces underscores with hyphens in user account. Changed collector code to handle this. (hygieia#2423)

* Corrected logic to evaluate commits that are on the PR but outside of queried date range.

* Github replaces underscores with hyphen in user accounts. Changed code to handle this.

* Fixing a minor error on the unit test that was not apparent, due to date when I tested it last

*  Update README (hygieia#2095)

Update README to describe how to start a mocked version of the UI to run the tests against

* Performance Hot House changes (hygieia#2401)

* Hothouse changes for performance evaulator

* Fongo unit test for performance evaulator

* cleanup for test data

* update branch with master

* code cleanup

* code cleanup

* Code clean up

* bug fix in performance evaluator

* Fixed the Audit Status for 'error' to PEER_REVIEW_LGTM_ERROR from PEER_REVIEW_LGTM_PENDING

* PT audit changes - Generic collector for Performance tests

* Sonar collector - null check for project data (hygieia#2436)

* recursively chown home

* missed user:group

* Update Readme

Updated to announce the Hygieia Org Move.

* Revert "Merge pull request hygieia#2431 from battinasubodh/master": To avoid downstream impacts at this point. (hygieia#2438)

This reverts commit 7e7cc87, reversing
changes made to c4f329d.

* fix locale to the US for formatted numbers (hygieia#2430)

* Added ability to globally publish build data using Jenkins Hygieia plugin. (hygieia#2396)

* Added ability to globally publish all build data.

* Rename variable to indicate that it is a global setup.

* Added global publish of Sonar result. Also added bunch of test cases.

* upgraded the hygieia publisher plugin version to 2.1.0 (hygieia#2440)

* Updated readme (hygieia#2441)

* updated readme

* dumy build to push repo

* Adding new item to model

* Remove instance variables from the listener. Changed test cases accordingly.

* Version increment for hygieia jenkins plugin (hygieia#2445)

* Updating model to revert change and add comment

* Adding commonName field compare

*  Publish events from Jenkins Pipeline to Hygieia and increment version. (hygieia#2452)

* Version increment for hygieia jenkins plugin

* Publish events from Jenkins Pipeline to Hygieia.

* Publish events from Jenkins Pipeline to Hygieia and increment version.

* Remove SpringBeans

* Removing some whitespaces and proper indendation

* Increasing Version to 2.1.3-SNAPSHOT

* Enhanced Jira feature collector with issueLinks (hygieia#2448)

* issue links details for Feature Collector

* collecting IssueLinks data for a Jira Issue

collecting IssueLinks data for a Jira Issue

* adding IssueLinkType to IssueLinks in Feature collection

* adding comments

* cleanup for issueLinkType data in feature collection

* removing unused import

* removing the _id part from the issueLinks

* fixing banner

* quick fix for the unit tests for fetaure collector

* unit tests for issueLinks

* updates to test data

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Give the github-graphql docker image a unique name, Fixes hygieia#2459 (hygieia#2460)

* Widget display on lastUpdated  (hygieia#2458)

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - get last updated test run  (hygieia#2467)

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* Widgets - get lastupdated widget to display. PT test results, remove testJob names in widget options

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - Get lastupdated testrun

* PT Audit changes - NPE check for test list

* Sca verbose bug 21485 (hygieia#2450)

* bug fix for `SCA Verbose Message not returning SCA Project URL consistently`

* cleanup

* covering for new coverage

* updating comments

* reverting items.json

* updating items.json

* addressing Nireesh comment for replacing nested catch with multi catch block

* SCA Unit Tests

* code cleanup

* New Feature: Auto complete dashboard by correlating events (hygieia#2466)

* Merge from upstream (#1)

* Java changes for General Configuration

* Java changes for general configuration

* Rally collector initial commit

* Fix for clearing servers in collectors collection

* Removing remaining days logic

* Fixing Remaining days in API layer rally controller

* Final fix for rallyburndown

* Change in log file name

* Adding README.md for rally collector

* Rally URL change

* usernames and passwords typo change

* Rally username and password details

* Redirect README.md to gh-pages

* fix generics eror: type 1

cast each Evalutor to remove generic information to enable build on some jvm

* Hygieia publisher changes - publish to multiple hygieia instances

* Implementing timeOffset for Incidents and ChangeOrders, and using service name for indients when affectedItem is not available

* Added logic to consider the environment field value on hpsm, and also added the unit tests

* HpsmSettings, changed variable name, new name: incidentEnvironments. And brought down the log-level to debug for hpsmIncidentUpdates collector

* Fixing a minor error on the unit test that was not apparent, due to date when I tested it last

* Version increment for hygieia jenkins plugin (hygieia#2445)

* Updating model to revert change and add comment

* Adding commonName field compare

*  Publish events from Jenkins Pipeline to Hygieia and increment version. (hygieia#2452)

* Version increment for hygieia jenkins plugin

* Publish events from Jenkins Pipeline to Hygieia.

* Publish events from Jenkins Pipeline to Hygieia and increment version.

* Remove SpringBeans

* Removing some whitespaces and proper indendation

* Increasing Version to 2.1.3-SNAPSHOT

* Enhanced Jira feature collector with issueLinks (hygieia#2448)

* issue links details for Feature Collector

* collecting IssueLinks data for a Jira Issue

collecting IssueLinks data for a Jira Issue

* adding IssueLinkType to IssueLinks in Feature collection

* adding comments

* cleanup for issueLinkType data in feature collection

* removing unused import

* removing the _id part from the issueLinks

* fixing banner

* quick fix for the unit tests for fetaure collector

* unit tests for issueLinks

* updates to test data

* New features to sync dashboard based on collector item relations. New API to create Generic collector item

* New features to sync dashboard based on collector item relations. New API to create Generic collector item - adding missing files

* New features to sync dashboard based on collector item relations. New API to create Generic collector item - adding more missing files

* New features to sync dashboard based on collector item relations. Correct test classes.

* New feature to allow creation of tool specific collector item.
Changes to jenkins plugin. Removed MultiSCM support.

* New feature to allow creation of tool specific collector item.
Changes to jenkins plugin. Removed MultiSCM support.

* Reverting some changes to fix test cases.

* Refacrtoring, test updates.

* Added processing time flag for generic collector item. More clean up

* New repository method to look up by processing time.

* New repository method to look up by processing time.

* Updated to give ability to insert a Component into CMDB (hygieia#2472)

* Updates for api to insert new cmdb item

* Adding some comments

* Add Ability to skip jobs from publishing data through Global Configuration (hygieia#2477)

* Added ability to exclude job names

* Added ability to exclude job names

* merge from upstream

* fix mongoDB script interpretation issue (hygieia#2367)

* All tokens to be used for SonarQube authentication

Per the docs for SonarQube at https://docs.sonarqube.org/display/DEV/Web+API when using token based authentication then the token goes into the username part, followed by a colon and empty password.

The current code base does not allow a null password, it is expecting both a username and password, this PR corrects that.

See https://docs.sonarqube.org/display/DEV/Web+API

* Corrected operator precedence

* Set collectorItemId - DeployServiceImpl

*  Audit API - set `SCA CODE_QUALITY_AUDIT_OK` for Metric Status Ok/Warn (hygieia#2484)

* set SCA CODE_QUALITY_AUDIT_OK for Metric Status Ok/Warn

* SCA set CODE_QUALITY_AUDIT_OK when status is OK/Warn

* Nicename - Sonar collector (hygieia#2488)

* Nicename - Sonar collector

* Update pom.xml (hygieia#2427)

frontend-maven-plugin  issue

* NFRR Collector - Audit API URL update
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
sidbha007 pushed a commit that referenced this pull request Jul 2, 2019
Artifactory collector changes (hygieia#2808)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants