Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception when running s3cmd sync #1363

Open
banjoh opened this issue Jan 16, 2024 · 1 comment
Open

Exception when running s3cmd sync #1363

banjoh opened this issue Jan 16, 2024 · 1 comment

Comments

@banjoh
Copy link

banjoh commented Jan 16, 2024

Description

When copying files from one bucket to a local drive, I saw this exception

….
download: ‘s3://....' -> '/tmp/archives/....' (9422831 bytes in 0.0 seconds, 307.48 MB/s) [162 of 163]
download: 's3://....' -> '/tmp/archives/....' (9309 bytes in 0.0 seconds, 7.06 MB/s) [163 of 163]

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ)
  If the error persists, please report the
  following lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /usr/bin/s3cmd --access_key=kurl --secret_key=<redacted> --host=minio.minio --no-ssl --host-bucket=<redacted> sync s3://<redacted_bucket> /tmp/archives
Problem: <class 'TypeError: 'dict_keys' object is not subscriptable
S3cmd:   2.4.0
python:   3.12.1 (main, Dec  8 2023, 02:59:13) [GCC 13.2.0]
environment LANG=NOTSET

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 3627, in <module>
    rc = main()
         ^^^^^^
  File "/usr/bin/s3cmd", line 3524, in main
    rc = cmd_func(args)
         ^^^^^^^^^^^^^^
  File "/usr/bin/s3cmd", line 2163, in cmd_sync
    return cmd_sync_remote2local(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/s3cmd", line 1719, in cmd_sync_remote2local
    _set_local_filename(copy_pairs, destination_base, source_args, dir_cache)
  File "/usr/bin/s3cmd", line 1408, in _set_local_filename
    source_args[0] == remote_list[remote_list.keys()[0]].get(u'object_uri_str', ''):
                                  ~~~~~~~~~~~~~~~~~~^^^
TypeError: 'dict_keys' object is not subscriptable

Reproduction

I'm not yet able to write a reproducer that gets to that code path, but I looked at the code and traced the exception to this line which was written years ago. remote_list.keys() creates a dictionary view which cannot be indexed directly.

docker run --rm python:3.12 python -c 'd = {"t": 123}; print(d.keys()[0])'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: 'dict_keys' object is not subscriptable

You would need to construct a list first using list() then get the first value.

docker run --rm python:3.12 python -c 'd = {"t": 123}; print(list(d.keys())[0])'
t

Environment

S3cmd: 2.4.0
python: 3.12.1

@himani1910
Copy link

add a / at the end of /tmp/archives
/usr/bin/s3cmd --access_key=kurl --secret_key= --host=minio.minio --no-ssl --host-bucket= sync s3://<redacted_bucket> /tmp/archives/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants