bug: Step Functions - States.StringToJson error when used in ResultSelector for S3 getObject action

[bilki]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Whenever I use the StringToJson intrinsic function (https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html#asl-intrsc-func-json-manipulate) in the ResultSelector of a task calling the S3 getObject action (see steps to reproduce for the definition), the step function fails to execute that function with the messages:

2024-05-07T12:41:51.176 ERROR --- [ad-77 (eval)] l.s.s.a.c.eval_component   : Exception=AttributeError, Details=["'StreamingBody' object has no attribute 'strip'"] at '(StringToJson| {'name': (StatesFunctionName| {'name': 'tringToJso', 'function_type': <StatesFunctionNameType.StringToJson: 12>}, 'arg_list': (FunctionArgumentList| {'arg_list': [(FunctionArgumentJsonPath| {'_value': <botocore.response.StreamingBody object at 0xffff23890730>, '_json_path': '$.Body'}], 'size': 1}}'
2024-05-07T12:41:51.176 ERROR --- [ad-77 (eval)] l.s.s.a.c.eval_component   : Exception=AttributeError, Details=["'StreamingBody' object has no attribute 'strip'"] at '(PayloadBindingIntrinsicFunc| {'field': 'latest', 'src': 'States.StringToJson($.Body)', 'function': (StringToJson| {'name': (StatesFunctionName| {'name': 'tringToJso', 'function_type': <StatesFunctionNameType.StringToJson: 12>}, 'arg_list': (FunctionArgumentList| {'arg_list': [(FunctionArgumentJsonPath| {'_value': <botocore.response.StreamingBody object at 0xffff23890730>, '_json_path': '$.Body'}], 'size': 1}}}'
2024-05-07T12:41:51.176 ERROR --- [ad-77 (eval)] l.s.s.a.c.eval_component   : Exception=AttributeError, Details=["'StreamingBody' object has no attribute 'strip'"] at '(PayloadTmpl| {'payload_bindings': [(PayloadBindingIntrinsicFunc| {'field': 'latest', 'src': 'States.StringToJson($.Body)', 'function': (StringToJson| {'name': (StatesFunctionName| {'name': 'tringToJso', 'function_type': <StatesFunctionNameType.StringToJson: 12>}, 'arg_list': (FunctionArgumentList| {'arg_list': [(FunctionArgumentJsonPath| {'_value': <botocore.response.StreamingBody object at 0xffff23890730>, '_json_path': '$.Body'}], 'size': 1}}}]}'
2024-05-07T12:41:51.176 ERROR --- [ad-77 (eval)] l.s.s.a.c.eval_component   : Exception=AttributeError, Details=["'StreamingBody' object has no attribute 'strip'"] at '(ResultSelector| {'payload_tmpl': (PayloadTmpl| {'payload_bindings': [(PayloadBindingIntrinsicFunc| {'field': 'latest', 'src': 'States.StringToJson($.Body)', 'function': (StringToJson| {'name': (StatesFunctionName| {'name': 'tringToJso', 'function_type': <StatesFunctionNameType.StringToJson: 12>}, 'arg_list': (FunctionArgumentList| {'arg_list': [(FunctionArgumentJsonPath| {'_value': <botocore.response.StreamingBody object at 0xffff23890730>, '_json_path': '$.Body'}], 'size': 1}}}]}}'
2024-05-07T12:41:51.176  WARN --- [ad-77 (eval)] l.s.s.a.c.s.s.execute_stat : State Task encountered an unhandled exception that lead to a State.Runtime error.
2024-05-07T12:41:51.176 ERROR --- [ad-77 (eval)] l.s.s.a.c.eval_component   : Exception=FailureEventException, Error=States.Runtime, Details={"taskFailedEventDetails": {"error": "States.Runtime", "cause": "'StreamingBody' object has no attribute 'strip'"}} at '(StateTaskServiceAwsSdk| {'comment': None, 'input_path': (InputPath| {'input_path_src': '$'}, 'output_path': (OutputPath| {'output_path': '$.s3result.latest'}, 'state_entered_event_type': 'TaskStateEntered', 'state_exited_event_type': 'TaskStateExited', 'result_path': (ResultPath| {'result_path_src': '$.s3result'}, 'result_selector': (ResultSelector| {'payload_tmpl': (PayloadTmpl| {'payload_bindings': [(PayloadBindingIntrinsicFunc| {'field': 'latest', 'src': 'States.StringToJson($.Body)', 'function': (StringToJson| {'name': (StatesFunctionName| {'name': 'tringToJso', 'function_type': <StatesFunctionNameType.StringToJson: 12>}, 'arg_list': (FunctionArgumentList| {'arg_list': [(FunctionArgumentJsonPath| {'_value': <botocore.response.StreamingBody object at 0xffff23890730>, '_json_path': '$.Body'}], 'size': 1}}}]}}, 'retry': None, 'catch': None, 'timeout': (TimeoutSeconds| {'timeout_seconds': 99999999, 'is_default': None}, 'heartbeat': None, 'parameters': (Parameters| {'payload_tmpl': (PayloadTmpl| {'payload_bindings': [(PayloadBindingValue| {'field': 'Bucket', 'value': (PayloadValueStr| {'val': 'mybucket'}}, (PayloadBindingValue| {'field': 'Key', 'value': (PayloadValueStr| {'val': 'latest.json'}}]}}, 'name': 'Read latest.json from S3', 'state_type': <StateType.Task: 16>, 'continue_with': <localstack.services.stepfunctions.asl.component.state.state_continue_with.ContinueWithNext object at 0xffff799e7690>, 'resource': (ServiceResource| {'_region': '', '_account': '', 'resource_arn': 'arn:aws:states:::aws-sdk:s3:getObject', 'partition': 'aws', 'service_name': 'aws-sdk', 'api_name': 's3', 'api_action': 'getObject', 'condition': None}}'

Expected Behavior

This task step should return the getObject action response Body field string value converted into a JSON in the step result latest field.

How are you starting LocalStack?

Custom (please describe below)

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

Testcontainers (https://github.com/testcontainers/testcontainers-scala) with 3.4.0 image

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

Simply create a step function with the following definition, upload a file latest.json to an S3 bucket (change bucket and file key accordingly), then execute it.

{
  "Comment": "States.StringToJson error",
  "StartAt": "Read latest.json from S3",
  "States": {
    "Read latest.json from S3": {
      "Type": "Task",
      "End": true,
      "OutputPath": "$.s3result.latest",
      "ResultPath": "$.s3result",
      "ResultSelector": {
        "latest.$": "States.StringToJson($.Body)"
      },
      "Resource": "arn:states:::aws-sdk:s3:getObject",
      "Parameters": {
        "Bucket": "bucket",
        "Key": "filekey"
      }
    }
  }
}

Environment

- OS: MacOS
- LocalStack: 3.4.0

Anything else?

No response

[localstack-bot]

Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines and our contributing guide.

[MEPalma]

@bilki Thank you for taking the time to compile this report. We recently merged some changes that aim to address this issue. I would be grateful if you could test the new build at your earliest convenience and provide feedback on whether it resolves the problem you encountered. Thank you once again for bringing this issue forward!

[gerson24]

Hi @MEPalma,
I've been testing the latest version and it seems the issues with reading from S3 are resolved. However, it looks like there's a similar problem when writing to S3.

Here's the step I'm trying to execute:

{
  "StartAt": "Write manifest file",
  "States": {
    "Write manifest file": {
      "Type": "Task",
      "OutputPath": null,
      "ResultPath": null,
      "Resource": "arn:aws:states:::aws-sdk:s3:putObject",
      "Parameters": {
        "Bucket": "bucket",
        "Key.$": "States.Format('backfills/v1/{}/manifest.json', $.backfill.id)",
        "ContentType": "application/json",
        "Body": {
          "backupDateTime.$": "$.backfill.backupDateTime"
        }
      }
    }
  }
}

And the error I'm getting is:

[-1695 (eval)] l.s.s.a.c.eval_component : Exception=FailureEventException, Error=States.Runtime, Details={"taskFailedEventDetails": {"error": "States.Runtime", "cause": "'dict' object has no attribute 'tell'"}}

I'm using AWS CDK with TypeScript. Interestingly, if I build the Body with the following code: Body: { backupDateTime: '12345'} It doesn't work. However, if I change it to: Body: JSON.stringify({ backupDateTime: '12345' }) It works.

Thanks for your help!