Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Redis processor fails with HGETALL #2282

Closed
webfrank opened this issue Dec 7, 2023 · 5 comments · Fixed by #2597
Closed

Redis processor fails with HGETALL #2282

webfrank opened this issue Dec 7, 2023 · 5 comments · Fixed by #2597
Labels
needs investigation It looks as though have all the information needed but investigation is required processors Any tasks or issues relating specifically to processors

Comments

@webfrank
Copy link

webfrank commented Dec 7, 2023

Hi,
I was trying to use redid processor with command HGETALL to retrieve all the fields and values inside an hashset but the processor return always nothing (it prints a blank line in the log)

This is a sample test:

input:
  generate:
    count: 1
    mapping: root = ""

pipeline:
  processors:
  - redis:
      url: redis://localhost:6379
      retries: 0
      command: hgetall
      args_mapping: root = ["demo:list:whitelist"]
  - log:
      message: ${! meta() }
  - catch:
    - log:
        message: Not Found

and output is:

INFO Running main config from specified file       @service=benthos benthos_version=4.24.0 path=whitelist.yaml
INFO Listening for HTTP requests at: http://0.0.0.0:4195  @service=benthos
INFO Launching a benthos instance, use CTRL+C to close  @service=benthos
INFO {}                                            @service=benthos label="" path=root.pipeline.processors.1

INFO Pipeline has terminated. Shutting down the service  @service=benthos

From reds cli:

HGETALL demo:list:whitelist

1) "AA123BB"
2) "Car 1"
3) "CC456DD"
4) "Car 2"
5) "MPC7772"
6) "Volkswagen"

Using HGET command with key and field works as expected.
@Jeffail Jeffail added needs investigation It looks as though have all the information needed but investigation is required processors Any tasks or issues relating specifically to processors labels Dec 10, 2023
@1998sachin
Copy link

I am also facing the same issue with hgetall. The output is empty.

@Statkern15
Copy link

Is there any update on this issue? My current workaround is to use multiple GET commands but that is pretty ugly and doesn't scale very well.

@1998sachin
Copy link

1998sachin commented May 11, 2024 •
edited

@Statkern15 You can use lua script to execute hgetall command.

@Statkern15
Copy link

@Statkern15 You can use lua script to execute hgetall command.

that's and excellent workaround thank you.

@mkysel
Copy link
Contributor

mkysel commented May 16, 2024 •
edited

I tracked down the issue to type limitations in https://github.com/benthosdev/benthos/blob/279e25b03b2207b19804213c851ec386076f43a6/public/service/message.go#L262
PR coming up in the next few days.

@mkysel mkysel mentioned this issue May 17, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs investigation It looks as though have all the information needed but investigation is required processors Any tasks or issues relating specifically to processors
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Handle hash commands such as HGETALL in redis processor mkysel/benthos
5 participants
@Jeffail @webfrank @mkysel @1998sachin @Statkern15