feat(output): Add HTML outputs to Prowler #4005

pedrooot · 2024-05-15T06:46:41Z

Description

HTML outputs are back for all the providers.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…3986) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>

github-actions · 2024-05-15T09:29:50Z

You can check the documentation for this PR here -> SaaS Documentation

github-actions · 2024-05-16T14:43:29Z

You can check the documentation for this PR here -> SaaS Documentation

sergargar · 2024-05-16T14:54:50Z

Great job!! Please change the screenshot in the docs index and solve the kubernetes summary tables.

github-actions · 2024-05-16T15:08:52Z

You can check the documentation for this PR here -> SaaS Documentation

jfagoagas

Great work bringing back the HTML format @pedrooot 👏

I left some comments since I don't understand some parts of the code. We need to make some parts clearer and fix the failing tests.

jfagoagas · 2024-05-17T10:26:27Z

docs/tutorials/reporting.md

@@ -294,14 +294,16 @@ The following code is an example output of the [JSON-ASFF](https://docs.aws.amaz
 ???+ note
    Each finding is a `json` object within a list.

+### HTML
+
+The following photo is an example of the HTML output:


Suggested change

The following photo is an example of the HTML output:

The following image is an example of the HTML output:

jfagoagas · 2024-05-17T10:28:37Z

prowler/__main__.py

+            if "html" in mode:
+                add_html_footer(
+                    global_provider.output_options.output_filename,
+                    args.output_directory,


Suggested change

args.output_directory,

global_provider.output_options.output_directory,

Please modify this too in line 269.

jfagoagas · 2024-05-17T10:28:52Z

prowler/__main__.py

+                fill_html_overview_statistics(
+                    stats,
+                    global_provider.output_options.output_filename,
+                    args.output_directory,


Suggested change

args.output_directory,

global_provider.output_options.output_directory,

jfagoagas · 2024-05-17T10:30:02Z

prowler/lib/cli/parser.py

@@ -146,8 +146,8 @@ def __init_outputs_parser__(self):
            "-M",
            nargs="+",
            help="Output modes, by default csv and json-oscf are saved. When using AWS Security Hub integration, json-asff output is also saved.",
-            default=["csv", "json-ocsf"],
-            choices=["csv", "json-asff", "json-ocsf"],
+            default=["csv", "json-ocsf", "html"],


Please fix the tests related with this change.

jfagoagas · 2024-05-17T10:30:57Z

prowler/lib/outputs/compliance/compliance.py

+        if not isinstance(finding, FindingOutput):
+            check_id = finding.check_metadata.CheckID
+        else:
+            check_id = finding.check_id


Why this change? The HTML should get this from the finding object.

jfagoagas · 2024-05-17T10:31:57Z

prowler/lib/outputs/html/html.py

+    timestamp,
+)
+from prowler.lib.logger import logger
+from prowler.lib.outputs.compliance.compliance import get_check_compliance


This shouldn't be needed here, please get that information from the finding object.

jfagoagas · 2024-05-17T10:32:47Z

prowler/lib/outputs/html/html.py

+        parameters = sys.argv[1:]
+        for index, parameter in enumerate(parameters):
+            if (
+                parameter == "--kubeconfig-file"
+                and "/.kube/config" in parameters[index + 1]
+            ):
+                parameters[index + 1] = "~/.kube/config"


Why this? Please leave this function jus to fill the header, if you need to do some other changes pass do them in another place.

jfagoagas · 2024-05-17T10:33:46Z

prowler/lib/outputs/html/html.py

+        # It is not pretty but useful
+        # AWS_provider --> aws
+        # GCP_provider --> gcp
+        # Azure_provider --> azure


I think we need to do it for K8s right?

Forgot to update the comment but it's already done for Kubernetes

jfagoagas · 2024-05-17T10:35:29Z

prowler/lib/outputs/outputs.py

+                            fill_html(
+                                file_descriptors["html"], finding_output, output_options
+                            )
+                            file_descriptors["html"].write("")


Why is this needed?

When the report is written, first of all the html header is created and the method fill_html is called when creating the report.

I know but I mean about file_descriptors["html"].write("")

Fact, it's unneeded

github-actions · 2024-05-20T09:16:14Z

You can check the documentation for this PR here -> SaaS Documentation

github-actions · 2024-05-20T09:16:39Z

You can check the documentation for this PR here -> SaaS Documentation

github-actions · 2024-05-20T11:02:12Z

You can check the documentation for this PR here -> SaaS Documentation

github-actions · 2024-05-20T11:30:02Z

You can check the documentation for this PR here -> SaaS Documentation

codecov · 2024-05-20T11:46:30Z

Codecov Report

Attention: Patch coverage is 34.09091% with 87 lines in your changes are missing coverage. Please review.

Project coverage is 86.23%. Comparing base (53cfcff) to head (0e35629).
Report is 4 commits behind head on master.

Files	Patch %	Lines
prowler/lib/outputs/html/html.py	26.41%	78 Missing ⚠️
prowler/__main__.py	0.00%	5 Missing ⚠️
prowler/lib/outputs/outputs.py	33.33%	2 Missing ⚠️
prowler/lib/outputs/summary_table.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4005      +/-   ##
==========================================
- Coverage   86.65%   86.23%   -0.43%     
==========================================
  Files         777      777              
  Lines       24139    24292     +153     
==========================================
+ Hits        20918    20947      +29     
- Misses       3221     3345     +124

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jfagoagas

Great work @pedrooot 👏 HTML is back 🚀

pedrooot added 4 commits May 14, 2024 14:15

feat(output): add html output WIP

b7cb7c1

feat(html): add more providers

4a8c6a9

feat(output): add html for kubernetes

9ffef3d

feat(tests): add tests related with html output

ba4aeb5

pedrooot requested review from a team as code owners May 15, 2024 06:46

sergargar and others added 3 commits May 15, 2024 11:29

chore(safety): ignore pip vulnerability (#4007)

e75e44a

docs: update for HTML

d1b92c8

github-actions bot added documentation github_actions Pull requests that update GitHub Actions code labels May 15, 2024

Merge branch 'master' into PRWLR-3709-prowler-oss-bring-back-html

a56aa5d

resolve comments

dc9a567

jfagoagas requested changes May 17, 2024

View reviewed changes

pedrooot added 2 commits May 20, 2024 11:12

resolve comments

9895848

resolve comments

0f16805

jfagoagas self-requested a review May 20, 2024 10:44

resolve comments and fix tests

2da760e

fix tests

0e35629

jfagoagas approved these changes May 20, 2024

View reviewed changes

jfagoagas merged commit db29c75 into master May 20, 2024
10 of 12 checks passed

jfagoagas deleted the PRWLR-3709-prowler-oss-bring-back-html branch May 20, 2024 15:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(output): Add HTML outputs to Prowler #4005

feat(output): Add HTML outputs to Prowler #4005

pedrooot commented May 15, 2024 •

edited

github-actions bot commented May 15, 2024

github-actions bot commented May 16, 2024

sergargar commented May 16, 2024

github-actions bot commented May 16, 2024

jfagoagas left a comment

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 17, 2024

pedrooot May 20, 2024

jfagoagas May 20, 2024

pedrooot May 20, 2024

github-actions bot commented May 20, 2024

github-actions bot commented May 20, 2024

github-actions bot commented May 20, 2024

github-actions bot commented May 20, 2024

codecov bot commented May 20, 2024

jfagoagas left a comment

	The following photo is an example of the HTML output:
	The following image is an example of the HTML output:

	args.output_directory,
	global_provider.output_options.output_directory,

feat(output): Add HTML outputs to Prowler #4005

feat(output): Add HTML outputs to Prowler #4005

Conversation

pedrooot commented May 15, 2024 • edited

Description

License

github-actions bot commented May 15, 2024

github-actions bot commented May 16, 2024

sergargar commented May 16, 2024

github-actions bot commented May 16, 2024

jfagoagas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented May 20, 2024

github-actions bot commented May 20, 2024

github-actions bot commented May 20, 2024

github-actions bot commented May 20, 2024

codecov bot commented May 20, 2024

Codecov Report

jfagoagas left a comment

Choose a reason for hiding this comment

pedrooot commented May 15, 2024 •

edited