AWS Client Parameters are stored as a dictionary and throw an error when assigned to an S3 bucket through the UI

[mthatt]

First check

• I added a descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the Prefect documentation for this issue.
• I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

First I create an S3 block and an AWS Credentials block from the SDK. If I assign the credentials block to the S3 bucket in the UI, the Client Parameters are incorrectly stored as a dictionary. This leads to an error any time I try to use the S3 bucket I created.

I can see from the output of my loaded S3 bucket loaded_s3 = S3Bucket.load("s3-from-code") that aws_client_parameters is stored as a dictionary rather than an instance of AwsClientParameters as it should be:

AwsCredentials(region_name=None, profile_name=None, aws_access_key_id=None, aws_session_token=None, **aws_client_parameters={'config': None, 'verify': True, 'use_ssl': True, 'api_version': None, 'endpoint_url': None, 'verify_cert_path': None}**, aws_secret_access_key=None)

Which leads to TypeError: unhashable type: 'dict' whenever I try to work with the S3 bucket.

Reproduction

1. Create an S3 block from the SDK:

block = S3Bucket(bucket_name="se-demo-flow-code-store", bucket_path="se-demo-flow-code-store/weather/")
block.save(name="s3-from-code", overwrite=True)`

2. Create an AWS Credentials block from the SDK:

AwsCredentials(
    aws_access_key_id="access_key",
    aws_secret_access_key="secret_key",
    aws_session_token="session_token",
    region_name="us-east-2"
).save("aws-credentials-from-code", overwrite=True)`

3. Assign the credentials block to S3 in the UI
   

4. Load in S3 Bucket:

loaded_s3 = S3Bucket.load("s3-from-code")`

5. Try to copy an object to the S3 bucket:

loaded_s3.copy_object(
    "my_folder/notes.txt",
    "my_folder/notes_copy.txt"
)`

Error

Traceback (most recent call last):
  File "/Users/mihir/Documents/Code/create_aws_creds.py", line 18, in <module>
    loaded_s3.copy_object(
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/events/instrument.py", line 73, in inner
    raise exc
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/events/instrument.py", line 70, in inner
    return function(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 259, in coroutine_wrapper
    return call()
           ^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 432, in __call__
    return self.result()
           ^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 318, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 179, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 389, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect_aws/s3.py", line 1236, in copy_object
    s3_client = self.credentials.get_s3_client()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect_aws/credentials.py", line 180, in get_s3_client
    return self.get_client(client_type=ClientType.S3)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect_aws/credentials.py", line 171, in get_client
    return _get_client_cached(ctx=self, client_type=client_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mihir/anaconda3/envs/updated-prefect/lib/python3.11/site-packages/prefect_aws/credentials.py", line 122, in __hash__
    hash(self.aws_client_parameters),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'dict'

Versions

Version:             2.16.9
API version:         0.8.4
Python version:      3.11.8
Git commit:          083def52
Built:               Thu, Apr 4, 2024 3:11 PM
OS/Arch:             darwin/arm64
Profile:             default
Server type:         cloud

Additional context

No response

[paulusaptus]

I ran into the same problem (v 2.19.0). Downgrading to v2.16.8 solved the issue.

[extdponomarchuk]

I faced the same issue when changed the persistence result configuration from a S3 Block to a S3Bucket Block:
prefect config set PREFECT_DEFAULT_RESULT_STORAGE_BLOCK=s3-bucket/my-bucket

  File "/home/root/ve/lib/python3.11/site-packages/prefect_aws/credentials.py", line 121, in __hash__
    hash(self.aws_client_parameters),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'dict'