Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cockpit pcp python #20049

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Cockpit pcp python #20049

wants to merge 8 commits into from

Conversation

jelly
Copy link
Member

@jelly jelly commented Feb 16, 2024 •
edited

Rewrite cockpit-pcp as a Python bridge channel using the python3-pcp bindings:

ToDo's when picking this up:

  • multi-instance support for meta message
  • Test different metric types
  • Test / support omit-instances
  • Test receiving multiple metrics
  • Test non-default interval
  • Drop cockpit-pcp from pkg/pcp/manifest.json and test the metrics page

The first version should support:

Source:

  • pcp-archive or / - this is the simplest case (used on metrics page)
  • direct - needs investigation
  • pmcd - needs investigation*

For the metrics page we need to support multi-instanced instances and a limit, timestamp and setting interval options. Including omit-instances support for network.interface.total.bytes.

Comparing to the C implementation:

cpf open metrics1 source="/tmp/pytest-of-jelle/pytest-current/instances-archives0/" metrics='[{ "name": "kernel.all.load" }]'  limit=1 : wait | G_MESSAGES_DEBUG=none /usr/lib/cockpit/cockpit-pcp | /usr/bin/cat

We also have some valid test archives in test/verify/files/metric-archives, untar them and you can then for example load them with:

[jelle@t14s][~/projects/cockpit/cockpit-pcp-python]%cpf open metrics1 source="/home/jelle/projects/cockpit/cockpit-pcp-python/var/log/pcp/pmlogger/localhost.localdomain/" metrics='[{ "name": "kernel.all.load" }]'  limit=100 : wait | G_MESSAGES_DEBUG=none /usr/lib/cockpit/cockpit-pcp | /usr/bin/cat
36

{ "command": "init", "version": 1 }36

{"command":"ready","channel":"ch1"}154
ch1
{"timestamp":1597663539413,"now":1713453024135,"interval":1000,"metrics":[{"name":"kernel.all.load","instances":[],"units":"","semantics":"instant"}]}10
ch1
[[[]]]187
ch1
{"timestamp":1597663540413,"now":1713453024135,"interval":1000,"metrics":[{"name":"kernel.all.load","instances":["15 minute","1 minute","5 minute"],"units":"","semantics":"instant"}]}360
ch1
[[[0.019999999552965164,0.23999999463558197,0.05000000074505806]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]]]264
ch1
[[[]],[[0.05000000074505806,0.52999997138977051,0.15000000596046448]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]],[[]]]36

@jelly jelly added the no-test For doc/workflow changes, or experiments which don't need a full CI run, label Feb 16, 2024
github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 16, 2024
View reviewed changes
src/cockpit/channels/pcp.py Fixed Show fixed Hide fixed
src/cockpit/channels/pcp.py
raise

@staticmethod
def semantic_val(sem_id: int):

Check notice

Code scanning / CodeQL

Explicit returns mixed with implicit (fall through) returns Note

Mixing implicit and explicit returns may indicate an error as implicit returns always return None.
src/cockpit/channels/pcp.py
# TODO: take care of this later...
if units:
try:
_pm_units_buf = context.pmParseUnitsStr(units)

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable _pm_units_buf is not used.
src/cockpit/channels/pcp.py
name=name,
derive=derive,
desc=pm_desc,
factor=factor,

Check failure

Code scanning / CodeQL

Potentially uninitialized local variable Error

Local variable 'factor' may be used before it is initialized.
@jelly jelly force-pushed the cockpit-pcp-python branch 2 times, most recently from 2821ebd to 920768d Compare February 21, 2024 11:53
github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 21, 2024
View reviewed changes
src/cockpit/channels/pcp.py
Comment on lines +240 to +241
# if self.omit_instances and self.instances:
# raise ChannelError('protocol-error', message='')

Check notice

Code scanning / CodeQL

Commented-out code Note

This comment appears to contain commented-out code.
src/cockpit/channels/pcp.py
Comment on lines +304 to +305
# if value_count > 1:
# _, instances = context.pmGetInDom(indom=descs[i].contents.indom)

Check notice

Code scanning / CodeQL

Commented-out code Note

This comment appears to contain commented-out code.
src/cockpit/channels/pcp.py
Comment on lines +309 to +321
# for j in range(value_count):
# atom = context.pmExtractValue(results.contents.get_valfmt(i),
# results.contents.get_vlist(i, j),
# content_type,
# content_type)
#
# if value_count > 1:
# assert isinstance(instances, list)
# assert isinstance(values, dict)
# values[instances[j]] = atom.dref(content_type)
# else:
# # TODO does float() need to be here?
# values = float(atom.dref(content_type))

Check notice

Code scanning / CodeQL

Commented-out code Note

This comment appears to contain commented-out code.
src/cockpit/channels/pcp.py

return samples

def build_sample(self, context, valfmt, value, desc):

Check notice

Code scanning / CodeQL

Explicit returns mixed with implicit (fall through) returns Note

Mixing implicit and explicit returns may indicate an error as implicit returns always return None.
src/cockpit/channels/pcp.py
except Exception as exc:
print("BORK", exc)

sample_value = atom.d

Check failure

Code scanning / CodeQL

Potentially uninitialized local variable Error

Local variable 'atom' may be used before it is initialized.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
no-test For doc/workflow changes, or experiments which don't need a full CI run,
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@jelly