Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ops Agent fails intermittently to execute type modify_fields instruction #1699

Open
PawelKGalka opened this issue May 10, 2024 · 1 comment

Comments

@PawelKGalka
Copy link

Describe the bug
Havin exemplary field in jsonPayload.auditLog.rawData.txTimeMillis it is not always parsed to integer in Logs Explorer and remains as string

To Reproduce
Steps to reproduce the behavior:

  1. Start a GCE VM with image RHEL 8
  2. Install Ops Agent version 2.46.1
  3. Customize the agent with configuration
logging:
  receivers:
    messages:
      type: files
      include_paths:
      - /var/log/messages
    secure:
      type: files
      include_paths:
      - /var/log/secure
    applogs_tcp:
      type: tcp
      format: json
      listen_port: 5160
  processors:
    parse:
      type: parse_json
      time_key: jsonPayload.timestamp
      time_format: "%Y-%m-%dT%H:%M:%S.%L%Z"
    adjust_fields_to_stackdriver:
      type: modify_fields
      fields:
        jsonPayload.auditLog.rawData.txTimeMillis:
          type: integer
  service:
    pipelines:
      default_pipeline:
        receivers: [messages, secure]
      applogs_pipeline:
        receivers: [applogs_tcp]
        processors: [parse, adjust_fields_to_stackdriver]

and restart the agent
5. See some values are still in String format
image

Expected behavior
Value is in integer format always

Environment (please complete the following information):

  • Project ID - can send via email
  • VM ID - can send via email
  • VM distro / OS: RHEL8
  • Ops Agent version 2.46.1
  • Ops Agent configuration
  • Ops Agent log

Additional context
NA

@PawelKGalka PawelKGalka changed the title Ops Agent fails sometime to execute type modify_fields instruction Ops Agent fails intermittently to execute type modify_fields instruction May 10, 2024
@jefferbrecht
Copy link
Member

Hi @PawelKGalka, we recently released Ops Agent 2.47.0 which upgrades the version of Fluent Bit used internally. Are you able to upgrade to the latest version and reproduce the issue still?

If you're still able to reproduce this, can you provide any sample input logs that you're sending to that TCP port so that we can try to reproduce this ourselves? Feel free to open a Cloud Support case as well which would let us troubleshoot much more closely (you can reference this GitHub issue in that case if you'd like).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants