Log messages exceeding size limits lost

[jgtr-blis]

Describe the bug
When parsing long/multiline log entries (usually due to stack traces and debug information) they often exceed the maximum size resulting in the message being rejected with the following error:

[2022/10/20 00:29:28] [ warn] [output:stackdriver:stackdriver.0] error
{
"error": {
"code": 400,
"message": "Log entry with size 679.5K exceeds maximum size of 256.0K",
"status": "INVALID_ARGUMENT"
}
}

When using a look ahead regex expression with the parse_regex processor there isn't an easy way that I'm aware of to limit the size of the message.

To Reproduce
Steps to reproduce the behaviour:

1. Create a log receiver with a parse_regex processor with a lookahead expression such as the following:
   ^(?<time>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2},\d{3}) (?<severity>[^ ]*) (?<msg>[\s\S]*?)(?=\g<time>|\Z)
   2.Read in a log entry that exceeds the 256KB limit:

2022-10-20T00:22:29,900 INFO [main] Test message1
2022-10-20T00:22:29,941 INFO [main] **** PAD THIS OUT WITH A 256KB MULTILINE PAYLOAD ***
2022-10-20T00:22:29,969 INFO [main] Test message2

Expected behaviour
A message should be written to stackdriver to indicate that a log had been dropped rather than it just be silently lost and/or a processor should be provided to allow specific fields in the payload to be truncated based on a maximum length so we can at least get a log entry and part of the message.

[braydonk]

Hi @jgtr-blis thanks for opening an issue. I am actively working on this. I don't know if we have a public issue tracker for it at the moment, but I will keep this issue updated with progress.

[marcdif]

Hi @braydonk, is this issue still being worked on? I've been running into a similar issue for some time now. However we're only using "parse_json" and "modify_fields", not the "parse_regex_ processor OP mentioned.

[2022/12/07 15:03:29] [ warn] [output:stackdriver:stackdriver.0] error
{
  "error": {
    "code": 400,
    "message": "Log entry with size 273.8K exceeds maximum size of 256.0K",
    "status": "INVALID_ARGUMENT"
  }
}

[2022/12/07 15:03:29] [ info] [engine] flush backlog chunk '1777-1670442648.114319951.flb' succeeded: task_id=29, input=storage_backlog.5 > output=stackdriver.0 (out_id=0)
[2022/12/07 15:03:29] [ warn] [output:stackdriver:stackdriver.0] error
{
  "error": {
    "code": 400,
    "message": "Log entry with size 396.4K exceeds maximum size of 256.0K",
    "status": "INVALID_ARGUMENT"
  }
}