Ops Agent does not support traceSampled field mapping

[PawelKGalka]

Describe the bug
When trying to set jsonPayload."logging.googleapis.com/trace_sampled" it is not displayed as mapped in Logs Explorer and remains in jsonPayload

To Reproduce
Steps to reproduce the behavior:

1. Start a GCE VM with image RHEL8
2. Install Ops Agent version 2.46.1
3. Customize the agent with configuration

logging:
  receivers:
    messages:
      type: files
      include_paths:
      - /var/log/messages
    secure:
      type: files
      include_paths:
      - /var/log/secure
    applogs_tcp:
      type: tcp
      format: json
      listen_port: 5160
  processors:
    parse:
      type: parse_json
      time_key: jsonPayload.timestamp
      time_format: "%Y-%m-%dT%H:%M:%S.%L%Z"
    adjust_fields_to_stackdriver:
      type: modify_fields
      fields:
        jsonPayload."logging.googleapis.com/trace_sampled":
          default_value: false
  service:
    pipelines:
      default_pipeline:
        receivers: [messages, secure]
      applogs_pipeline:
        receivers: [applogs_tcp]
        processors: [parse, adjust_fields_to_stackdriver]

and restart the agent
6. See log entry does not have traceSampled property and it remains in jsonPayload as jsonPayload."logging.googleapis.com/trace_sampled": "false"

Expected behavior
Value mapped in logs explorer

Environment (please complete the following information):

• Project ID - can send via email
• VM ID - can send via email
• VM distro / OS: RHEL8
• Ops Agent version 2.46.1
• Ops Agent configuration
• Ops Agent log

Additional context
NA

[braydonk]

Hi @PawelKGalka,

Can you try doing logging.googleapis.com/traceSampled (instead of trace_sampled) and see if that works?

[PawelKGalka]

hi @braydonk
No luck with above, I was referring to Google documentation on structured logging:
https://cloud.google.com/logging/docs/structured-logging

We were facing similar issue with logging.googleapis.com/spanId field in same manner but I got resolved I think with some release of ops agent so potentially this needs similar development. Also we are unable to send it as boolean in jsonPayload to map so would it work if we use map_values mapping?

map_values:
  "true" : true
  "false": false

[jefferbrecht]

Hi @PawelKGalka, just to learn a bit more about your use case, would it be acceptable if the Ops Agent were to set traceSampled automatically based on whether there's a spanId and trace present?

We've had a few other folks ask for ability to set traceSampled manually as well, and it seems like all use cases so far would benefit from something automatic instead so that you don't need to configure anything.

[PawelKGalka]

Hi @jefferbrecht,

Our application is utilizing tracing in 3 modes based on particular criterias - DROP,RECORD,RECORD_AND_SAMPLE (from OTEL lib). This means there are cases where we trace to correlate transaction in the logs but we do not sample it to Cloud Trace. So for our use case it would be best to have this configurable in the config. Or it can take precedence over default setting when spanId/traceId is set